-- Logs begin at Fri 2022-08-19 20:13:28 UTC, end at Fri 2022-08-19 21:33:37 UTC. -- Aug 19 20:53:19.191193 dv-cinder-852209-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 19 20:53:19.205494 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 19 20:53:19.208635 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Aug 19 20:53:19 2022] *** Aug 19 20:53:19.208635 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 19 20:53:19.208635 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 19 20:53:19.208635 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: nodename: dv-cinder-852209-6-iscsi Aug 19 20:53:19.208635 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: machine: x86_64 Aug 19 20:53:19.208635 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: clock source: unix Aug 19 20:53:19.208635 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: pcre jit disabled Aug 19 20:53:19.208635 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: detected number of CPU cores: 4 Aug 19 20:53:19.209202 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: current working directory: / Aug 19 20:53:19.209202 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: detected binary path: /usr/bin/uwsgi-core Aug 19 20:53:19.209202 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: your processes number limit is 47680 Aug 19 20:53:19.209202 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: your memory page size is 4096 bytes Aug 19 20:53:19.209202 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: detected max file descriptor number: 2048 Aug 19 20:53:19.209202 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: lock engine: pthread robust mutexes Aug 19 20:53:19.209202 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: thunder lock: enabled Aug 19 20:53:19.209202 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 19 20:53:19.209664 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 19 20:53:19.278770 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: Python main interpreter initialized at 0x55f5a77019c0 Aug 19 20:53:19.278770 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: python threads support enabled Aug 19 20:53:19.278770 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: your server socket listen backlog is limited to 100 connections Aug 19 20:53:19.278770 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: your mercy for graceful operations on workers is 90 seconds Aug 19 20:53:19.279316 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: mapped 403077 bytes (393 KB) for 2 cores Aug 19 20:53:19.279469 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: *** Operational MODE: preforking *** Aug 19 20:53:19.279948 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: *** uWSGI is running in multiple interpreter mode *** Aug 19 20:53:19.279948 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: spawned uWSGI master process (pid: 81565) Aug 19 20:53:19.280215 dv-cinder-852209-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 19 20:53:19.285805 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: spawned uWSGI worker 1 (pid: 81570, cores: 1) Aug 19 20:53:19.285805 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: spawned uWSGI worker 2 (pid: 81571, cores: 1) Aug 19 20:53:19.285805 dv-cinder-852209-6-iscsi devstack@keystone.service[81565]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 19 20:53:23.760205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 20:53:23.760205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: from cryptography.utils import int_from_bytes Aug 19 20:53:23.761392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 20:53:23.761392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: from cryptography.utils import int_from_bytes Aug 19 20:53:23.812859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 19 20:53:23.813130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 19 20:53:23.813260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] command line args: [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 19 20:53:23.814026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 19 20:53:23.814211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ================================================================================ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 19 20:53:23.814477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] admin_token = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.814731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_dir = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.814923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.815113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_source = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.815296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] debug = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.815501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.815906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.816081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.816335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.816590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.816828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.817009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_config_append = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.817178 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.817344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_dir = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.817505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.817752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.818027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.818281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.818536 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.818708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.818875 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.819126 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.819380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.819621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.819859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.820096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.820357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.820591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.820847 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.821132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.821646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.821848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] publish_errors = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.822088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.822325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.822574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.822787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.822988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.823220 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] standard_threads = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.823455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.823698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.823918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.824098 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_journal = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.824271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_json = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.824403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_stderr = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.824595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_syslog = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.824761 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:23.825030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.825273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.825538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.825803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.826057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.826300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.826554 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.826795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.827048 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.827298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.827553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.827791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.828043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.828284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.828537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.828791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.829096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.829887 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.830151 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.830454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.830637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.830895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.831188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.831422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.831676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.831912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.832148 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.832383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.832624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.832878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.833130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.833366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.833601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.833878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.834137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.834379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.834631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.834871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.835111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.835356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.835593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.835829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.836068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.836305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.836545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.836802 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.837039 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.837280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.837517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.837788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.838023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.838279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.838513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.838756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.838999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.839241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.839482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.839724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.839957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.840213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.840453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.840695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.840934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.841200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.841439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.841677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.841946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.842188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.842467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.842710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.842964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.843215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.843456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.843694 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.843934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.844189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.844431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.844939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.845071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.845251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.845492 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.845853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.846096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.846340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.846611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.846855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.847112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.847334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.847576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.847823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.848064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.848303 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.848539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.848776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.849012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.849253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.849487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.849735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.850004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.850262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.850503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.850765 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.851031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.851275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.851513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.external = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.851748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.852052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.852287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.852525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.password = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.852759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.token = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.853008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.853244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.853500 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.853732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.854009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.854267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.854504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.854743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.854992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.855248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.855498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.855737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.855984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.856222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.856479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.856735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.856973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.857231 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.857523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.857845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.858060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.858310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.858572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.858814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.859038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.859288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.859540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.859773 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.860025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.860908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.861320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.861999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.861999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.862280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.862404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.862608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.862915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.863120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.863363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.863607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.863859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.864097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.864341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.864601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.864858 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.865095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.865331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.865569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.865842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.866096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.866351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.866612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.866863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.867318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.867535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.867716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.868113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.868321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.868556 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.868797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.869044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.869290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.869543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.869813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.870048 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.870303 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.870544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.870958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.871213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.871449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.871694 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.871931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.872175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.872418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.872679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.872888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.873135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.873390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.873644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.873906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.874145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.874399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.874655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.874895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.875138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.875377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.875615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.876772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.876772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.877737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.877737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.877737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.877737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.878262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.878262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.878702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.878702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.878971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.878971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.879239 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.879492 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.879744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.879996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.880247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.880481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.880735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.880979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.881222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.881463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.881704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.881981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.882234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.882480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.882737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.882973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.883222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.883465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.883704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.883946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.884196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.884449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.884685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.884928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.885175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.885429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.885683 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.885938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.886188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.886445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.886685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.886938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.887179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.887423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.887662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.887917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.888150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.888406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.888660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.888913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.889166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.889420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.889669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.889941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.890195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.890432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.890686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.890942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.891195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.891808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.892146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.892146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.892376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.892779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.892983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.893170 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.893357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.893589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.893873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.894126 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.894340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.894574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.894811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.895067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.895328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.895580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.895820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.896062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.896299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.896540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.896781 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.897037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.897276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.897538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.897817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.898058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.898307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.898555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.898806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.899047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.899290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.899529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.899782 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.900034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.900275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.900501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.900810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.901079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.901326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.901570 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.901829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.902075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.902320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.902560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.902803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.903045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.903295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.903537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.903787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.904028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.904287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.904529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.904780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.905019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.905268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.905506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:23.905705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 19 20:53:23.937234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:23.937589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:23.937858 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:23.947211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:23.976636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:23.977004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.037450 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.091458 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.091707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.261539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.261956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.262223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.262223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.262609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.262609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.288521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.288826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.289050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.289050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.289391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.289579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.289817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.302438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.315893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.316302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.316302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.316703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.316935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.316935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.317317 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.334604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.334948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 20:53:24.335242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.335464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 20:53:24.335782 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.335991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.336427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.336695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.347006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.347301 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.347582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.347875 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.355066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.358055 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.358347 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.358635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.358924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.359213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.359481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.359758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.360034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.361158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.361478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.361614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.385963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.387624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.387938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.393361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.393660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.393977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.400512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.400837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.404970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.405263 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.405553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.405874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.406137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.406417 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.406693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.406967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.424947 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.425241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.429256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.432577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.432881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.437255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.441074 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.444385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.444690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.444979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.445277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.445561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.445888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.462643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.462968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.463228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.463501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.471320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.472943 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.473222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.473375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 20:53:24.473375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: from cryptography.utils import int_from_bytes Aug 19 20:53:24.474283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 19 20:53:24.474283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: from cryptography.utils import int_from_bytes Aug 19 20:53:24.475981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.476279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.476575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.476882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.477163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.489658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.489990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.490280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.490569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.490857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.491139 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.491423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.508109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.511293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.514241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.515704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.517284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.517664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.517991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.526156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.528991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.531563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 19 20:53:24.531757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 19 20:53:24.531944 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] command line args: [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 19 20:53:24.533147 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.533324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 19 20:53:24.533324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ================================================================================ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 19 20:53:24.533556 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.533692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] admin_token = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.533871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.533871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.534147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_dir = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.534147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.534147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_source = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.534498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] debug = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.534625 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.534799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.534969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.535239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.535493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.535728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.535911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_config_append = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.536192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.536393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_dir = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.536523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.536678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.536936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.537196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.537451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.537631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.537822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.538079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.538329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.538630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.538801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.539151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.539606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.539955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.540217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.540502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.541029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.541214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] publish_errors = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.541456 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.541692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.541899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.542155 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.542396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.542636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] standard_threads = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.542872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.543122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.543299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.543433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.543567 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_journal = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.543699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.543844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_json = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.543844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_stderr = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.544144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.544277 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_syslog = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.544277 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 19 20:53:24.544515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.544659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.544934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.545177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.545428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.545671 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.545953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.546200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.546450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.546702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.546959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.547199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.547451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.547740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.548507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.548767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.549001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.549300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.549643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.549940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.550182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.550444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.550713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.550971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.551226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.551485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.551686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.551848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.551988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.552149 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.552285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.552431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.552562 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.552691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.552818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.552946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.553085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.553217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.553335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.553457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.553580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.553711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.553886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.554018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.554144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.554274 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.554274 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.554508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.554651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.554778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.554909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.555038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.555163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.555294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.555424 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.555549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.555674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.555674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.555886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.556097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.556332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.556570 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.556803 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.557042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.557281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.557533 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.558114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.558114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.558331 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.558479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.558740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.558955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.559189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.559442 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.559682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.559936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.560172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.560438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.560679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.560916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.561157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.561399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.561675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.561947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.562202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.562451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.562690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.562930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.563169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.563968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.564289 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.564727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.565115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.565364 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.565609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.565882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.566124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.566365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.566636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.566880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.567124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.567363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.567607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.567848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.568088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.568326 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.568559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.568792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.569015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.569249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.569482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.569725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.569989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.570242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.570481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.570741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.571004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.571248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.571482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.external = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.571716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.572015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.572249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.572483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.password = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.572728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.token = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.572971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.573210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.573459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.573694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.573969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.574220 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.574454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.574696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.574944 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.575196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.575447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.575691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.575938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.576176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.576452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.576709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.576950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.577207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.577499 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.577748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.578039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.578281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.578545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.578783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.579863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.579863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.580170 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.580299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.580597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.580947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.581206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.581447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.581706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.581974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.582227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.582466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.582715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.582967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.583204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.583442 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.583691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.583923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.584162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.584417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.584674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.584908 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.585142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.585378 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.585633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.585909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.586162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.586420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.586672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.586947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.587199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.587441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.587676 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.587910 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.588143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.588388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.588654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.588901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.589157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.589393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.589648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.589928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.590170 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.590424 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.590674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.590909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.591154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.591390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.591637 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.591875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.592124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.592365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.592606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.592863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.593115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.593351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.593585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.593994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.594250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.594497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.594754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.595109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.595350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.595677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.595994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.596236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.596494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.596748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.597001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.597242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.597479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.597720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.597998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.598254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.598493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.598749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.599001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.599252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.599504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.599755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.599990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.600245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.600487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.600730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.600851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f5a77019c0 pid: 81570 (default app) Aug 19 20:53:24.600970 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.601197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.601448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.601701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.601972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.602234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.602468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.602719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.603232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.603232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.603425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.603652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.603897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.604132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.604373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.604590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8501ec-8b26-4ae7-9941-8e072616af01 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:24.604725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.604860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.604974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8501ec-8b26-4ae7-9941-8e072616af01 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:24.605117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.605233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8501ec-8b26-4ae7-9941-8e072616af01 None None] PATH_INFO: `/v3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:24.605374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.605629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.605916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.606155 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.606405 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.606645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.606888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.607127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.607378 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.607611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.607865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.608114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.608366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.608616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.608865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.609114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.609368 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.609619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.609876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.612326 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.612573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Fri Aug 19 20:53:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:24.613569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.613569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.613569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.613569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.613569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.613569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.613569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.613569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.615083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.615083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.615083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.615083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.615083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.615083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.615083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.615083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.615879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.615879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.616082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.616245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.616732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.616732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.617018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.617203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.617450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.617693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.618263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.618516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.618757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.619005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.619253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.619507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.619746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.619989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.620231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.620483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.620777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.620967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.621230 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.621548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.621850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.622104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.622359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.622603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.622848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.623088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.623330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.623572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.623812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.624062 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.624302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.624549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.624785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.625041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.625279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.625539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.626463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.626608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.626906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 19 20:53:24.627083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 19 20:53:24.654727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.654944 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.655179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.664702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.684915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.685228 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.724079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.768141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.768356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.814740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.814957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.815103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.815308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.815530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.815749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.832805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.832805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.832805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.832805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.832805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.832805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.833572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.843850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.936923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.937164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.937331 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.937554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.937798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.938013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.938231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 19 20:53:24.949310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 20:53:24.949885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.950090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 19 20:53:24.950372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.950640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.950902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.951165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.962015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.962343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.962576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.962851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.970235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.973719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:24.973935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.974250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.974535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.974819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.975095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.975374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.975651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.975926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.976218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:24.976490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.000384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.002566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.002856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.007639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.007940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.008231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.015024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.015336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.020094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.020393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.020678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.020960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.021249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.021543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.021842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.022119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.038781 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.039078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.043065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.046222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.046522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.051702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.054929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.058126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.058432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.058730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.059019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.059303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.059606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.076009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.076235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.076471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.076739 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.083854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.085466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.085745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.090934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.090934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.090934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.091350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.091350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.103404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.103702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.103992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.104280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.104566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.104846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.105141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.123101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.127002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.130160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.131845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.133472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.133897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.134277 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.141083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.144036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.147367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.147763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.148147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.148527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.156914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.158040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.158460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.165675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 19 20:53:25.166107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.166479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.166886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.167250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.167620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.167987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.168354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.168836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.169216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.169585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.169973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.170347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.170707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 19 20:53:25.207460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f5a77019c0 pid: 81571 (default app) Aug 19 20:53:31.685341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623d9b5-8837-406a-9a97-8b4035e7124e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:31.686063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623d9b5-8837-406a-9a97-8b4035e7124e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:31.686063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623d9b5-8837-406a-9a97-8b4035e7124e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:31.687240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 2/2] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:53:31.696047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:31.696618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:31.696618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:31.779192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: INFO dbcounter [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] Registered counter for database keystone Aug 19 20:53:31.800353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG oslo_db.sqlalchemy.engines [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81571) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 19 20:53:31.807532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writer thread running {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 19 20:53:32.000165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG passlib.handlers.bcrypt [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81571) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 19 20:53:32.001227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG passlib.handlers.bcrypt [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81571) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 19 20:53:32.058392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:32.060090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:32.205902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1/3] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 19 20:53:32.651110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623d9b5-8837-406a-9a97-8b4035e7124e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:32.720220 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: INFO dbcounter [None req-b623d9b5-8837-406a-9a97-8b4035e7124e None None] Registered counter for database keystone Aug 19 20:53:32.742728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG oslo_db.sqlalchemy.engines [None req-b623d9b5-8837-406a-9a97-8b4035e7124e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81570) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 19 20:53:32.749650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writer thread running {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 19 20:53:32.876906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623d9b5-8837-406a-9a97-8b4035e7124e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:32.881711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2761c5-ae80-4fd8-b2aa-e74524d40399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:32.882422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2761c5-ae80-4fd8-b2aa-e74524d40399 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:32.882704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2761c5-ae80-4fd8-b2aa-e74524d40399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:32.882978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2761c5-ae80-4fd8-b2aa-e74524d40399 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:32.892773 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2761c5-ae80-4fd8-b2aa-e74524d40399 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:32.945833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1e2761c5-ae80-4fd8-b2aa-e74524d40399 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 19 20:53:32.946081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2761c5-ae80-4fd8-b2aa-e74524d40399 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:32.957611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 3/4] 10.222.0.16 () {58 vars in 1324 bytes} [Fri Aug 19 20:53:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 20:53:32.965556 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:32.985479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfcf0df-b813-4ebb-910b-d1b983e29259 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:32.990256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d747d0-d1fa-48fd-8532-8869b22252fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:32.991419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d747d0-d1fa-48fd-8532-8869b22252fe admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:32.991702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d747d0-d1fa-48fd-8532-8869b22252fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:32.991970 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d747d0-d1fa-48fd-8532-8869b22252fe admin admin] PATH_INFO: `/v3/projects/4f484db3eb0540218c8a18fc8601facf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:32.993803 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d747d0-d1fa-48fd-8532-8869b22252fe admin admin] RBAC: Authorizing `identity:get_project(project_id=4f484db3eb0540218c8a18fc8601facf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:33.046620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WARNING keystone.common.rbac_enforcer.enforcer [None req-72d747d0-d1fa-48fd-8532-8869b22252fe admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 19 20:53:33.046944 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d747d0-d1fa-48fd-8532-8869b22252fe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:33.052460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 2/5] 10.222.0.16 () {56 vars in 1207 bytes} [Fri Aug 19 20:53:32 2022] GET /identity/v3/projects/4f484db3eb0540218c8a18fc8601facf => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:34.991677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e73d4-3dd3-42c8-93e9-f72f1dbe0b7c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:34.992247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e73d4-3dd3-42c8-93e9-f72f1dbe0b7c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:34.992247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e73d4-3dd3-42c8-93e9-f72f1dbe0b7c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:34.996658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 4/6] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:53:35.004083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed362df0-01ad-4012-b91e-d14f8b8170d4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:35.004591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed362df0-01ad-4012-b91e-d14f8b8170d4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:35.004684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed362df0-01ad-4012-b91e-d14f8b8170d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:35.026011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ed362df0-01ad-4012-b91e-d14f8b8170d4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:35.027572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ed362df0-01ad-4012-b91e-d14f8b8170d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:35.036456 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 3/7] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 19 20:53:35.320268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e73d4-3dd3-42c8-93e9-f72f1dbe0b7c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:35.339992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e73d4-3dd3-42c8-93e9-f72f1dbe0b7c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:35.347664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8344a-093e-48c3-9828-03c789d2be16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:35.348694 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8344a-093e-48c3-9828-03c789d2be16 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:35.349194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8344a-093e-48c3-9828-03c789d2be16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:35.349673 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8344a-093e-48c3-9828-03c789d2be16 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:35.363465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8344a-093e-48c3-9828-03c789d2be16 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:35.366324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8344a-093e-48c3-9828-03c789d2be16 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:35.382380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 5/8] 10.222.0.16 () {58 vars in 1324 bytes} [Fri Aug 19 20:53:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 19 20:53:35.388170 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed362df0-01ad-4012-b91e-d14f8b8170d4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:35.399055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed362df0-01ad-4012-b91e-d14f8b8170d4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:35.403482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a690f-8f1f-4634-8aca-eb09edc30967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:35.404119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a690f-8f1f-4634-8aca-eb09edc30967 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:35.404397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a690f-8f1f-4634-8aca-eb09edc30967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:35.404668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a690f-8f1f-4634-8aca-eb09edc30967 admin admin] PATH_INFO: `/v3/users/3476c91ad03941919217e0940471d201` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:35.406031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a690f-8f1f-4634-8aca-eb09edc30967 admin admin] RBAC: Authorizing `identity:get_user(user_id=3476c91ad03941919217e0940471d201)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:35.408422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a690f-8f1f-4634-8aca-eb09edc30967 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:35.410052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 4/9] 10.222.0.16 () {56 vars in 1198 bytes} [Fri Aug 19 20:53:35 2022] GET /identity/v3/users/3476c91ad03941919217e0940471d201 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:53:37.466031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7db644-36d7-45c1-baeb-f994a2927cdf None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:37.466792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7db644-36d7-45c1-baeb-f994a2927cdf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:37.466946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7db644-36d7-45c1-baeb-f994a2927cdf None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:37.469922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 6/10] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:53:37.475883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153898-14e3-4ceb-b932-80e6cfc24bdb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:37.476656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153898-14e3-4ceb-b932-80e6cfc24bdb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:37.476656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f153898-14e3-4ceb-b932-80e6cfc24bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:37.497375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4f153898-14e3-4ceb-b932-80e6cfc24bdb None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:37.498737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4f153898-14e3-4ceb-b932-80e6cfc24bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:37.589291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529fe24-6aa4-4d68-bd49-a285cb7d7a2e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:37.589805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529fe24-6aa4-4d68-bd49-a285cb7d7a2e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:37.590328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c529fe24-6aa4-4d68-bd49-a285cb7d7a2e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:37.593084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 5/11] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:53:37.593372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 7/12] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:53:37.601701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a802f9d-80f0-4a01-b3fc-2cea6d8f59bf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:37.602175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a802f9d-80f0-4a01-b3fc-2cea6d8f59bf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:37.602567 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a802f9d-80f0-4a01-b3fc-2cea6d8f59bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:37.624248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6a802f9d-80f0-4a01-b3fc-2cea6d8f59bf None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:37.626515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6a802f9d-80f0-4a01-b3fc-2cea6d8f59bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:37.646913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:37.714469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 6/13] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:53:37.865600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529fe24-6aa4-4d68-bd49-a285cb7d7a2e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:38.009557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a802f9d-80f0-4a01-b3fc-2cea6d8f59bf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:38.043112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c529fe24-6aa4-4d68-bd49-a285cb7d7a2e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:38.098648 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f46707-0fd2-4c7e-afab-3a3bad266f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:38.099764 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f46707-0fd2-4c7e-afab-3a3bad266f39 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:38.100179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f46707-0fd2-4c7e-afab-3a3bad266f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:38.100540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f46707-0fd2-4c7e-afab-3a3bad266f39 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:38.116107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f46707-0fd2-4c7e-afab-3a3bad266f39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-65f46707-0fd2-4c7e-afab-3a3bad266f39 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:53:38.127843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:53:38.130188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:53:38.132033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 8/14] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:53:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:53:38.135768 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f46707-0fd2-4c7e-afab-3a3bad266f39 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:38.160038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a802f9d-80f0-4a01-b3fc-2cea6d8f59bf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:38.184099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:38.213703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239ba2a-00f8-400c-92bc-aea04ff5536a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:38.214719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239ba2a-00f8-400c-92bc-aea04ff5536a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:38.215098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239ba2a-00f8-400c-92bc-aea04ff5536a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:38.215509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239ba2a-00f8-400c-92bc-aea04ff5536a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:38.252027 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:38.291804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f46707-0fd2-4c7e-afab-3a3bad266f39 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:38.312316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239ba2a-00f8-400c-92bc-aea04ff5536a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:38.314273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239ba2a-00f8-400c-92bc-aea04ff5536a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:38.338721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fff2a-2ea3-41d0-99e7-0f3cbfdbe9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:38.339353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fff2a-2ea3-41d0-99e7-0f3cbfdbe9dd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:38.339636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fff2a-2ea3-41d0-99e7-0f3cbfdbe9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:38.339919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2fff2a-2ea3-41d0-99e7-0f3cbfdbe9dd None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:38.340613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2fff2a-2ea3-41d0-99e7-0f3cbfdbe9dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:38.340917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2fff2a-2ea3-41d0-99e7-0f3cbfdbe9dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:38.342398 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2fff2a-2ea3-41d0-99e7-0f3cbfdbe9dd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:38.355348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 9/15] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:53:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:53:38.361688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fff2a-2ea3-41d0-99e7-0f3cbfdbe9dd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:38.443805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 7/16] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 20:53:38.450072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239ba2a-00f8-400c-92bc-aea04ff5536a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:38.486067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:38.492667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2fff2a-2ea3-41d0-99e7-0f3cbfdbe9dd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:38.524949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:38.534963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa4208-a7f0-4f19-9606-3be3abbbf578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:38.535618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa4208-a7f0-4f19-9606-3be3abbbf578 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:38.535898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa4208-a7f0-4f19-9606-3be3abbbf578 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:38.536177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa4208-a7f0-4f19-9606-3be3abbbf578 None admin] PATH_INFO: `/v3/users/3476c91ad03941919217e0940471d201` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:38.537871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa4208-a7f0-4f19-9606-3be3abbbf578 None admin] RBAC: Authorizing `identity:get_user(user_id=3476c91ad03941919217e0940471d201)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:38.539358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa4208-a7f0-4f19-9606-3be3abbbf578 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:38.541003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 10/17] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:53:38 2022] GET /identity/v3/users/3476c91ad03941919217e0940471d201 => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:38.549839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa4208-a7f0-4f19-9606-3be3abbbf578 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:38.592856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239ba2a-00f8-400c-92bc-aea04ff5536a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:38.658549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2975a-b1ef-4524-a3ef-46ea7349c236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:38.659443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2975a-b1ef-4524-a3ef-46ea7349c236 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:38.659821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2975a-b1ef-4524-a3ef-46ea7349c236 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:38.660197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2975a-b1ef-4524-a3ef-46ea7349c236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:38.676031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa4208-a7f0-4f19-9606-3be3abbbf578 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:38.680956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2975a-b1ef-4524-a3ef-46ea7349c236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:38.683879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2975a-b1ef-4524-a3ef-46ea7349c236 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:38.717091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0b4c-7159-4daf-932a-f70998396fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:38.717692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0b4c-7159-4daf-932a-f70998396fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:38.718014 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0b4c-7159-4daf-932a-f70998396fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:38.718292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0b4c-7159-4daf-932a-f70998396fa3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:38.719815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a0b4c-7159-4daf-932a-f70998396fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-37f2975a-b1ef-4524-a3ef-46ea7349c236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:53:38.719990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:53:38.721315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:53:38.722723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:53:38.722723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:53:38.722723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:53:38.723045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a0b4c-7159-4daf-932a-f70998396fa3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:38.723175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 8/18] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:53:38.723473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 11/19] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:53:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:53:38.729362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2975a-b1ef-4524-a3ef-46ea7349c236 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:38.731683 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0b4c-7159-4daf-932a-f70998396fa3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:38.789304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:38.855155 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2975a-b1ef-4524-a3ef-46ea7349c236 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:38.858317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:38.881519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0b4c-7159-4daf-932a-f70998396fa3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:38.911311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a8039f-1a4d-4eec-b395-b439bccd2c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:38.912246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8039f-1a4d-4eec-b395-b439bccd2c32 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:38.912708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8039f-1a4d-4eec-b395-b439bccd2c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:38.913162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8039f-1a4d-4eec-b395-b439bccd2c32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:38.914498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8039f-1a4d-4eec-b395-b439bccd2c32 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.domain.id=default, user.id=3476c91ad03941919217e0940471d201) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:38.915006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8039f-1a4d-4eec-b395-b439bccd2c32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:38.917490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a8039f-1a4d-4eec-b395-b439bccd2c32 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:38.928348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 9/20] 10.222.0.16 () {56 vars in 1322 bytes} [Fri Aug 19 20:53:38 2022] GET /identity/v3/role_assignments?user.id=3476c91ad03941919217e0940471d201&scope.domain.id=default&role.id=8b350845f331423cb0494e3449b58310 => generated 233 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:38.930709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c08bd-f433-4e80-ad65-6722f35c83c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:38.931466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c08bd-f433-4e80-ad65-6722f35c83c2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:38.931834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c08bd-f433-4e80-ad65-6722f35c83c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:38.932204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c08bd-f433-4e80-ad65-6722f35c83c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:38.932991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c08bd-f433-4e80-ad65-6722f35c83c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:38.933405 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c08bd-f433-4e80-ad65-6722f35c83c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:38.935144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c08bd-f433-4e80-ad65-6722f35c83c2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:38.962404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 12/21] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:41.542085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be85ca-888a-4a77-b7e3-0d8671ac8aba None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:41.542916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be85ca-888a-4a77-b7e3-0d8671ac8aba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:41.542916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be85ca-888a-4a77-b7e3-0d8671ac8aba None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:41.544011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 10/22] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:53:41.550218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157a801-956c-4c10-8e43-991561284a4e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:41.550929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157a801-956c-4c10-8e43-991561284a4e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:41.550929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157a801-956c-4c10-8e43-991561284a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:41.592876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG passlib.handlers.bcrypt [None req-c157a801-956c-4c10-8e43-991561284a4e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81570) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 19 20:53:41.593304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG passlib.handlers.bcrypt [None req-c157a801-956c-4c10-8e43-991561284a4e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81570) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 19 20:53:41.652246 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c157a801-956c-4c10-8e43-991561284a4e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:41.653924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c157a801-956c-4c10-8e43-991561284a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:41.820001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 13/23] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 19 20:53:42.378038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb880-afc7-4306-b1a4-cb81768a9ede None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:42.378258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb880-afc7-4306-b1a4-cb81768a9ede None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:42.378611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bb880-afc7-4306-b1a4-cb81768a9ede None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:42.379876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 11/24] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:53:42.387644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63850666-80dd-4032-87e9-fb46a4b77c30 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:42.388022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63850666-80dd-4032-87e9-fb46a4b77c30 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:42.388273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63850666-80dd-4032-87e9-fb46a4b77c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:42.393078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bb880-afc7-4306-b1a4-cb81768a9ede None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:42.423594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-63850666-80dd-4032-87e9-fb46a4b77c30 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:42.424857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-63850666-80dd-4032-87e9-fb46a4b77c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:42.455048 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:42.568570 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 14/25] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:53:42.641955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bb880-afc7-4306-b1a4-cb81768a9ede None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:42.694239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781420c-5362-4bee-a9ca-a991d4b67963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:42.694889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781420c-5362-4bee-a9ca-a991d4b67963 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:42.695171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781420c-5362-4bee-a9ca-a991d4b67963 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:42.695447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781420c-5362-4bee-a9ca-a991d4b67963 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:42.696162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781420c-5362-4bee-a9ca-a991d4b67963 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:42.698614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781420c-5362-4bee-a9ca-a991d4b67963 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:42.838911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 12/26] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Aug 19 20:53:42 2022] POST /identity/v3/roles => generated 219 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 20:53:43.081750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff49fdc-b6ae-4597-a52e-1d1bbe7133c5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.081750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff49fdc-b6ae-4597-a52e-1d1bbe7133c5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.081750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff49fdc-b6ae-4597-a52e-1d1bbe7133c5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.081750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 15/27] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:53:43.096878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769f2a6-a8ed-4f83-9ff0-17e82bd9ee3f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.097163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769f2a6-a8ed-4f83-9ff0-17e82bd9ee3f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.097608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769f2a6-a8ed-4f83-9ff0-17e82bd9ee3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.128780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e769f2a6-a8ed-4f83-9ff0-17e82bd9ee3f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:43.130670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e769f2a6-a8ed-4f83-9ff0-17e82bd9ee3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:43.136638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767545a2-f5a9-4e47-86c2-7806a327352b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.136923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767545a2-f5a9-4e47-86c2-7806a327352b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.137203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767545a2-f5a9-4e47-86c2-7806a327352b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.138646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 16/28] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:53:43.144602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47aca6-a9a2-480a-89b2-8c8254c5b778 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.145917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47aca6-a9a2-480a-89b2-8c8254c5b778 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.146422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c47aca6-a9a2-480a-89b2-8c8254c5b778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.172645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:43.175591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8c47aca6-a9a2-480a-89b2-8c8254c5b778 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:43.177082 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8c47aca6-a9a2-480a-89b2-8c8254c5b778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:43.278417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 13/29] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:53:43.295095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769f2a6-a8ed-4f83-9ff0-17e82bd9ee3f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:43.318213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 17/30] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:53:43.455914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769f2a6-a8ed-4f83-9ff0-17e82bd9ee3f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:43.536668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fd88d-059e-4358-bee0-d3b71fd17116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:43.537721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fd88d-059e-4358-bee0-d3b71fd17116 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.538259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fd88d-059e-4358-bee0-d3b71fd17116 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.538755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928fd88d-059e-4358-bee0-d3b71fd17116 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.568839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928fd88d-059e-4358-bee0-d3b71fd17116 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-928fd88d-059e-4358-bee0-d3b71fd17116 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:53:43.581682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:53:43.582599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:53:43.583348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:53:43.590339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 14/31] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:53:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:53:43.593131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47aca6-a9a2-480a-89b2-8c8254c5b778 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:43.646141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fd88d-059e-4358-bee0-d3b71fd17116 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:43.667324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:43.772011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928fd88d-059e-4358-bee0-d3b71fd17116 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:43.800243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:43.814906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabb3286-bb5d-41c2-a850-319a4baaae57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:43.815387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb3286-bb5d-41c2-a850-319a4baaae57 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.815671 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb3286-bb5d-41c2-a850-319a4baaae57 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.815946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb3286-bb5d-41c2-a850-319a4baaae57 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.816644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb3286-bb5d-41c2-a850-319a4baaae57 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:43.818246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabb3286-bb5d-41c2-a850-319a4baaae57 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:43.823370 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c47aca6-a9a2-480a-89b2-8c8254c5b778 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:43.841233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 15/32] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Aug 19 20:53:43 2022] POST /identity/v3/roles => generated 225 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 20:53:43.842860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2eb721-07fa-4ffc-a9be-dcc62fe19777 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.843346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2eb721-07fa-4ffc-a9be-dcc62fe19777 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.843757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2eb721-07fa-4ffc-a9be-dcc62fe19777 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.850156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 16/33] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:53:43.850156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b10ea-c6c5-4455-a0dc-6876ade1ef2d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.850156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b10ea-c6c5-4455-a0dc-6876ade1ef2d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.850156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b10ea-c6c5-4455-a0dc-6876ade1ef2d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.850156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 17/34] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:53:43.854462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf948dad-06ee-4d23-a841-6c4cedbc0527 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.855073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf948dad-06ee-4d23-a841-6c4cedbc0527 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.855492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf948dad-06ee-4d23-a841-6c4cedbc0527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.887404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cf948dad-06ee-4d23-a841-6c4cedbc0527 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:43.888854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cf948dad-06ee-4d23-a841-6c4cedbc0527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:43.901380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ab43b2-f693-4d42-a95a-b9b002a3e4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:43.902301 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab43b2-f693-4d42-a95a-b9b002a3e4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.902740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab43b2-f693-4d42-a95a-b9b002a3e4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.903128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ab43b2-f693-4d42-a95a-b9b002a3e4d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.903974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab43b2-f693-4d42-a95a-b9b002a3e4d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:43.904391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab43b2-f693-4d42-a95a-b9b002a3e4d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:43.906418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ab43b2-f693-4d42-a95a-b9b002a3e4d9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:43.917426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 18/35] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:53:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:43.919080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed691-c465-4731-be9d-d2b6a95af20f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:43.919495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed691-c465-4731-be9d-d2b6a95af20f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:43.919896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ed691-c465-4731-be9d-d2b6a95af20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:43.942705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bc5ed691-c465-4731-be9d-d2b6a95af20f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:43.943755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc5ed691-c465-4731-be9d-d2b6a95af20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:44.040102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 18/36] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:53:44.040889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf948dad-06ee-4d23-a841-6c4cedbc0527 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:44.053432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:44.060932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 19/37] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:53:44.063853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ed691-c465-4731-be9d-d2b6a95af20f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:44.191634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf948dad-06ee-4d23-a841-6c4cedbc0527 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:44.230314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:44.236751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52331abb-fb51-4f91-8190-ce3771b84f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:44.237733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52331abb-fb51-4f91-8190-ce3771b84f31 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:44.238278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52331abb-fb51-4f91-8190-ce3771b84f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:44.238763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52331abb-fb51-4f91-8190-ce3771b84f31 None admin] PATH_INFO: `/v3/users/3476c91ad03941919217e0940471d201` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:44.241103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ed691-c465-4731-be9d-d2b6a95af20f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:44.241292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52331abb-fb51-4f91-8190-ce3771b84f31 None admin] RBAC: Authorizing `identity:get_user(user_id=3476c91ad03941919217e0940471d201)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:44.245066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52331abb-fb51-4f91-8190-ce3771b84f31 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:44.250748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 19/38] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:53:44 2022] GET /identity/v3/users/3476c91ad03941919217e0940471d201 => generated 247 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:44.263714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52331abb-fb51-4f91-8190-ce3771b84f31 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:44.288407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747d79e-5f08-4a90-8dd5-664a037a5a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:44.289168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d79e-5f08-4a90-8dd5-664a037a5a69 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:44.289515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d79e-5f08-4a90-8dd5-664a037a5a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:44.289936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747d79e-5f08-4a90-8dd5-664a037a5a69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:44.314204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747d79e-5f08-4a90-8dd5-664a037a5a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:44.315804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747d79e-5f08-4a90-8dd5-664a037a5a69 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6747d79e-5f08-4a90-8dd5-664a037a5a69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:53:44.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:53:44.342829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:53:44.343592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:53:44.343592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:53:44.343592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:53:44.343592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 20/39] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:53:44.354224 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747d79e-5f08-4a90-8dd5-664a037a5a69 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:44.373115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:44.399290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52331abb-fb51-4f91-8190-ce3771b84f31 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:44.437080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8af14-d45a-4f22-95db-5d0e22f503bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:44.437720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8af14-d45a-4f22-95db-5d0e22f503bd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:44.438599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8af14-d45a-4f22-95db-5d0e22f503bd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:44.439076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8af14-d45a-4f22-95db-5d0e22f503bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:44.440728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8af14-d45a-4f22-95db-5d0e22f503bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:44.442245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8af14-d45a-4f22-95db-5d0e22f503bd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:44.443963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 20/40] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:53:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:44.446699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8af14-d45a-4f22-95db-5d0e22f503bd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:44.470328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747d79e-5f08-4a90-8dd5-664a037a5a69 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:44.499765 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:44.514702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64790d-136e-47c6-ad6e-28684fd6fd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:44.515745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64790d-136e-47c6-ad6e-28684fd6fd84 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:44.516901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64790d-136e-47c6-ad6e-28684fd6fd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:44.517390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64790d-136e-47c6-ad6e-28684fd6fd84 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:44.518582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64790d-136e-47c6-ad6e-28684fd6fd84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:44.519126 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64790d-136e-47c6-ad6e-28684fd6fd84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:44.521885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64790d-136e-47c6-ad6e-28684fd6fd84 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:44.537127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 21/41] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:53:44.538172 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64790d-136e-47c6-ad6e-28684fd6fd84 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:44.583250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8af14-d45a-4f22-95db-5d0e22f503bd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:44.620198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d733955-97b0-4aad-9364-5356b9253e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:44.620809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d733955-97b0-4aad-9364-5356b9253e23 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:44.621088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d733955-97b0-4aad-9364-5356b9253e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:44.621363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d733955-97b0-4aad-9364-5356b9253e23 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:44.622064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d733955-97b0-4aad-9364-5356b9253e23 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:44.623460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d733955-97b0-4aad-9364-5356b9253e23 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:44.644738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 21/42] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Aug 19 20:53:44 2022] POST /identity/v3/roles => generated 223 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:53:44.644963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d733955-97b0-4aad-9364-5356b9253e23 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:44.650077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:44.654595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64790d-136e-47c6-ad6e-28684fd6fd84 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:44.715761 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73525e8c-905f-41d0-a788-a0004a5e513e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:44.716431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525e8c-905f-41d0-a788-a0004a5e513e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:44.716890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525e8c-905f-41d0-a788-a0004a5e513e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:44.717390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73525e8c-905f-41d0-a788-a0004a5e513e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:44.720478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73525e8c-905f-41d0-a788-a0004a5e513e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:44.723330 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73525e8c-905f-41d0-a788-a0004a5e513e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:44.726605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 22/43] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:53:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:44.727832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73525e8c-905f-41d0-a788-a0004a5e513e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:44.788984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d733955-97b0-4aad-9364-5356b9253e23 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:44.796131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:44.847815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9f85b-9d60-434a-a3c6-598715159837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:44.848812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9f85b-9d60-434a-a3c6-598715159837 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:44.849238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9f85b-9d60-434a-a3c6-598715159837 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:44.849661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9f85b-9d60-434a-a3c6-598715159837 None admin] PATH_INFO: `/v3/domains/default/users/3476c91ad03941919217e0940471d201/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:44.853696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9f85b-9d60-434a-a3c6-598715159837 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3476c91ad03941919217e0940471d201, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:44.855094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9f85b-9d60-434a-a3c6-598715159837 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:44.868434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73525e8c-905f-41d0-a788-a0004a5e513e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:44.878763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 22/44] 10.222.0.16 () {58 vars in 1361 bytes} [Fri Aug 19 20:53:44 2022] PUT /identity/v3/domains/default/users/3476c91ad03941919217e0940471d201/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:53:44.880529 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9f85b-9d60-434a-a3c6-598715159837 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:44.924044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d565975b-5a35-4558-a4da-aaa354a9b9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:44.925185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565975b-5a35-4558-a4da-aaa354a9b9f7 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:44.925622 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565975b-5a35-4558-a4da-aaa354a9b9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:44.926190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565975b-5a35-4558-a4da-aaa354a9b9f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:44.927455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565975b-5a35-4558-a4da-aaa354a9b9f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:44.930653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d565975b-5a35-4558-a4da-aaa354a9b9f7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:44.969835 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:45.040057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9f85b-9d60-434a-a3c6-598715159837 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:45.087599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37516fa8-e2a5-49ff-963b-be294fe6d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:45.088212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37516fa8-e2a5-49ff-963b-be294fe6d15d None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:45.088493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37516fa8-e2a5-49ff-963b-be294fe6d15d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:45.088765 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37516fa8-e2a5-49ff-963b-be294fe6d15d None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:45.089550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37516fa8-e2a5-49ff-963b-be294fe6d15d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:45.091030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37516fa8-e2a5-49ff-963b-be294fe6d15d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:45.097433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 23/45] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:53:44 2022] POST /identity/v3/projects => generated 301 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:53:45.128929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 23/46] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:53:44 2022] POST /identity/v3/projects => generated 312 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:53:46.921124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c397e7-6ad8-4e19-93ae-84f27123fd47 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:46.921706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c397e7-6ad8-4e19-93ae-84f27123fd47 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:46.921953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c397e7-6ad8-4e19-93ae-84f27123fd47 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:46.924456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 24/47] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:53:46.930136 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276d096-29ac-435e-9a88-85feef08515b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:46.930659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276d096-29ac-435e-9a88-85feef08515b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:46.930772 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276d096-29ac-435e-9a88-85feef08515b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:46.950211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f276d096-29ac-435e-9a88-85feef08515b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:46.951427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f276d096-29ac-435e-9a88-85feef08515b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:47.024665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 24/48] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:53:47.135506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352e658-de68-44e5-a0d8-0e0cd48f3faa None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:47.135796 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352e658-de68-44e5-a0d8-0e0cd48f3faa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:47.136080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352e658-de68-44e5-a0d8-0e0cd48f3faa None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:47.137449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 25/49] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:53:47.143098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab5fc-38a2-4278-948f-372795947541 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:47.143384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab5fc-38a2-4278-948f-372795947541 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:47.143660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab5fc-38a2-4278-948f-372795947541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:47.162665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ce6ab5fc-38a2-4278-948f-372795947541 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:47.163878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ce6ab5fc-38a2-4278-948f-372795947541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:47.212712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:47.256242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 25/50] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:53:47.322605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352e658-de68-44e5-a0d8-0e0cd48f3faa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:47.513721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352e658-de68-44e5-a0d8-0e0cd48f3faa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:47.528259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:47.543357 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ab5fc-38a2-4278-948f-372795947541 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:47.570265 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06411b5-4f47-4117-941f-f36d86044c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:47.571117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06411b5-4f47-4117-941f-f36d86044c58 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:47.571488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06411b5-4f47-4117-941f-f36d86044c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:47.571852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06411b5-4f47-4117-941f-f36d86044c58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:47.586439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06411b5-4f47-4117-941f-f36d86044c58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a06411b5-4f47-4117-941f-f36d86044c58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:53:47.591748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:53:47.593411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:53:47.594888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 26/51] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:53:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:53:47.605257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06411b5-4f47-4117-941f-f36d86044c58 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:47.661673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ab5fc-38a2-4278-948f-372795947541 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:47.717948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109fd61-f5b8-4133-937f-2c530f721ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:47.719015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109fd61-f5b8-4133-937f-2c530f721ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:47.719427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109fd61-f5b8-4133-937f-2c530f721ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:47.719829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8109fd61-f5b8-4133-937f-2c530f721ab2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:47.721500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109fd61-f5b8-4133-937f-2c530f721ab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:47.723180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8109fd61-f5b8-4133-937f-2c530f721ab2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:47.724969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 26/52] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:53:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:53:47.731450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109fd61-f5b8-4133-937f-2c530f721ab2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:47.789644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:47.871324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8109fd61-f5b8-4133-937f-2c530f721ab2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:47.910150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06411b5-4f47-4117-941f-f36d86044c58 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:47.930201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58347860-537d-48ad-b4b7-5c6ec218d685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:47.930911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58347860-537d-48ad-b4b7-5c6ec218d685 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:47.931117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58347860-537d-48ad-b4b7-5c6ec218d685 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:47.931393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58347860-537d-48ad-b4b7-5c6ec218d685 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:47.932061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58347860-537d-48ad-b4b7-5c6ec218d685 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:47.933575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58347860-537d-48ad-b4b7-5c6ec218d685 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:47.953516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd53ab-bdf4-45f7-b870-e23168662274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:47.954268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd53ab-bdf4-45f7-b870-e23168662274 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:47.954667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd53ab-bdf4-45f7-b870-e23168662274 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:47.954835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd53ab-bdf4-45f7-b870-e23168662274 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:47.955555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd53ab-bdf4-45f7-b870-e23168662274 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:47.955861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd53ab-bdf4-45f7-b870-e23168662274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:47.957484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd53ab-bdf4-45f7-b870-e23168662274 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:47.965936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 27/53] 10.222.0.16 () {60 vars in 1140 bytes} [Fri Aug 19 20:53:47 2022] POST /identity/v3/projects => generated 298 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 20:53:47.973412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 27/54] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:53:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:53:47.975963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58347860-537d-48ad-b4b7-5c6ec218d685 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:48.103399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:48.111576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58347860-537d-48ad-b4b7-5c6ec218d685 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:48.158932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff769b-7ea5-455f-ab3a-21c914e82e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:48.159657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff769b-7ea5-455f-ab3a-21c914e82e69 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:48.160071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff769b-7ea5-455f-ab3a-21c914e82e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:48.160430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fff769b-7ea5-455f-ab3a-21c914e82e69 None admin] PATH_INFO: `/v3/users/3476c91ad03941919217e0940471d201` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:48.165363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff769b-7ea5-455f-ab3a-21c914e82e69 None admin] RBAC: Authorizing `identity:get_user(user_id=3476c91ad03941919217e0940471d201)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:48.165363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fff769b-7ea5-455f-ab3a-21c914e82e69 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:48.165363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 28/55] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:53:47 2022] GET /identity/v3/users/3476c91ad03941919217e0940471d201 => generated 247 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:48.174237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd53ab-bdf4-45f7-b870-e23168662274 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:48.216676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:48.345665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd53ab-bdf4-45f7-b870-e23168662274 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:48.407971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8ea59-d9c6-483b-aaaf-ee33f6f55bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:48.409060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8ea59-d9c6-483b-aaaf-ee33f6f55bde None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:48.409555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8ea59-d9c6-483b-aaaf-ee33f6f55bde None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:48.410061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8ea59-d9c6-483b-aaaf-ee33f6f55bde None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:48.412283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8ea59-d9c6-483b-aaaf-ee33f6f55bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:48.415035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8ea59-d9c6-483b-aaaf-ee33f6f55bde None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:48.418067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 28/56] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:53:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:53:48.425748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff769b-7ea5-455f-ab3a-21c914e82e69 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:48.603919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fff769b-7ea5-455f-ab3a-21c914e82e69 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:48.645576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00244fe-53de-4df5-a8ff-f319c5445e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:48.646396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00244fe-53de-4df5-a8ff-f319c5445e21 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:48.646753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00244fe-53de-4df5-a8ff-f319c5445e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:48.647105 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00244fe-53de-4df5-a8ff-f319c5445e21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:48.647952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00244fe-53de-4df5-a8ff-f319c5445e21 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.domain.id=default, user.id=3476c91ad03941919217e0940471d201) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:48.648322 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00244fe-53de-4df5-a8ff-f319c5445e21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:48.649992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00244fe-53de-4df5-a8ff-f319c5445e21 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:48.657619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 29/57] 10.222.0.16 () {56 vars in 1322 bytes} [Fri Aug 19 20:53:48 2022] GET /identity/v3/role_assignments?user.id=3476c91ad03941919217e0940471d201&scope.domain.id=default&role.id=8b350845f331423cb0494e3449b58310 => generated 530 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:50.184207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0b25-fd6f-4cb2-969e-c2149aae365c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:50.184724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0b25-fd6f-4cb2-969e-c2149aae365c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:50.184809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e0b25-fd6f-4cb2-969e-c2149aae365c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:50.186164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 29/58] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:53:50.192591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b659767-fadb-4f64-b49c-1ef51e93de21 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:50.193091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b659767-fadb-4f64-b49c-1ef51e93de21 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:50.193226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b659767-fadb-4f64-b49c-1ef51e93de21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:50.211713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9b659767-fadb-4f64-b49c-1ef51e93de21 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:50.212949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9b659767-fadb-4f64-b49c-1ef51e93de21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:50.238152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:50.288012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 30/59] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:53:50.561944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0b25-fd6f-4cb2-969e-c2149aae365c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:50.671623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:50.680520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e0b25-fd6f-4cb2-969e-c2149aae365c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:50.723603 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a74f5b-f217-45d7-a171-35ac3b6bb145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:50.724268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a74f5b-f217-45d7-a171-35ac3b6bb145 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:50.724546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a74f5b-f217-45d7-a171-35ac3b6bb145 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:50.724823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a74f5b-f217-45d7-a171-35ac3b6bb145 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:50.726088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a74f5b-f217-45d7-a171-35ac3b6bb145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:50.727577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a74f5b-f217-45d7-a171-35ac3b6bb145 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:50.729257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 30/60] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:53:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:50.735284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b659767-fadb-4f64-b49c-1ef51e93de21 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:50.879082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b659767-fadb-4f64-b49c-1ef51e93de21 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:50.919440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87c1ebe-a6b9-4a51-aa2d-bf991263bdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:50.920063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c1ebe-a6b9-4a51-aa2d-bf991263bdfc None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:50.920342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c1ebe-a6b9-4a51-aa2d-bf991263bdfc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:50.920614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87c1ebe-a6b9-4a51-aa2d-bf991263bdfc None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:50.921284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c1ebe-a6b9-4a51-aa2d-bf991263bdfc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:50.923753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87c1ebe-a6b9-4a51-aa2d-bf991263bdfc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:50.956314 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 31/61] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:53:50 2022] POST /identity/v3/users => generated 275 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 20:53:54.484370 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ff02f-27ef-4aab-a817-0232c0011f32 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:54.486531 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ff02f-27ef-4aab-a817-0232c0011f32 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:54.487022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ff02f-27ef-4aab-a817-0232c0011f32 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:54.488640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 31/62] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:53:54.494928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3e453-90a9-4183-a652-409d241377d3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:54.495850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3e453-90a9-4183-a652-409d241377d3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:54.496356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf3e453-90a9-4183-a652-409d241377d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:54.526891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cbf3e453-90a9-4183-a652-409d241377d3 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:54.528486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cbf3e453-90a9-4183-a652-409d241377d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:54.545977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:54.639638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 32/63] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:53:54.784638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5301d-6115-40f2-89e4-1ef74144338c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:54.785277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5301d-6115-40f2-89e4-1ef74144338c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:54.785749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b5301d-6115-40f2-89e4-1ef74144338c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:54.787541 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 32/64] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:53:54.793513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187f8d-90c6-4c3e-915a-72935871b112 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:54.793950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187f8d-90c6-4c3e-915a-72935871b112 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:54.794379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62187f8d-90c6-4c3e-915a-72935871b112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:54.821434 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-62187f8d-90c6-4c3e-915a-72935871b112 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:54.822683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-62187f8d-90c6-4c3e-915a-72935871b112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:54.916482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a4a4-52e2-4c52-ab48-fa50f8eae512 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:54.917075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a4a4-52e2-4c52-ab48-fa50f8eae512 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:54.917554 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2a4a4-52e2-4c52-ab48-fa50f8eae512 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:54.919148 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 33/65] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:53:54.921508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 33/66] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:53:54.925811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603db4-82da-4c6d-bcb2-107ad95b0c37 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:54.926395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603db4-82da-4c6d-bcb2-107ad95b0c37 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:54.926899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603db4-82da-4c6d-bcb2-107ad95b0c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:54.942815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465abf87-2c3c-4944-ae56-ccfc969534aa None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:54.943144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465abf87-2c3c-4944-ae56-ccfc969534aa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:54.943381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465abf87-2c3c-4944-ae56-ccfc969534aa None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:54.944805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 34/67] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:53:54.956721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf3974-b118-4ddc-8c5e-e7eff6f5fd36 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:54.957230 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf3974-b118-4ddc-8c5e-e7eff6f5fd36 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:54.957660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf3974-b118-4ddc-8c5e-e7eff6f5fd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:54.960197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f6603db4-82da-4c6d-bcb2-107ad95b0c37 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:54.965816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f6603db4-82da-4c6d-bcb2-107ad95b0c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:54.992547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5ddf3974-b118-4ddc-8c5e-e7eff6f5fd36 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:54.994058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5ddf3974-b118-4ddc-8c5e-e7eff6f5fd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:55.070037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 34/68] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:53:55.071642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603db4-82da-4c6d-bcb2-107ad95b0c37 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:55.094620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 35/69] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:53:55.192560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:55.200034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603db4-82da-4c6d-bcb2-107ad95b0c37 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:55.217612 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf3974-b118-4ddc-8c5e-e7eff6f5fd36 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:55.222503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:55.278339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd654a-817f-4e21-96a2-e03871ef4e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:55.279005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd654a-817f-4e21-96a2-e03871ef4e11 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:55.279285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd654a-817f-4e21-96a2-e03871ef4e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:55.279558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd654a-817f-4e21-96a2-e03871ef4e11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:55.305646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd654a-817f-4e21-96a2-e03871ef4e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-11dd654a-817f-4e21-96a2-e03871ef4e11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:53:55.310677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:53:55.311521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:53:55.312259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 35/70] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:53:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:53:55.319933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd654a-817f-4e21-96a2-e03871ef4e11 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:55.421709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf3974-b118-4ddc-8c5e-e7eff6f5fd36 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:55.463865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86511cbc-be4e-452e-bda9-987ce431231c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:55.464643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86511cbc-be4e-452e-bda9-987ce431231c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:55.465036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86511cbc-be4e-452e-bda9-987ce431231c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:55.465324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86511cbc-be4e-452e-bda9-987ce431231c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:55.472062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd654a-817f-4e21-96a2-e03871ef4e11 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:55.481105 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86511cbc-be4e-452e-bda9-987ce431231c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-86511cbc-be4e-452e-bda9-987ce431231c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:53:55.489411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:53:55.490561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 36/71] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:53:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:53:55.494494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86511cbc-be4e-452e-bda9-987ce431231c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:55.522853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62b77-5445-4275-80ae-04219d54d7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:55.523580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62b77-5445-4275-80ae-04219d54d7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:55.523850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62b77-5445-4275-80ae-04219d54d7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:55.524127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62b77-5445-4275-80ae-04219d54d7d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:55.524882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62b77-5445-4275-80ae-04219d54d7d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:55.525195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62b77-5445-4275-80ae-04219d54d7d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:55.526772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62b77-5445-4275-80ae-04219d54d7d3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:55.537549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 36/72] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:53:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:55.539021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62b77-5445-4275-80ae-04219d54d7d3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:55.565242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:55.671566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62b77-5445-4275-80ae-04219d54d7d3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:55.695751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:55.715950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1628fa-81b0-4255-ab2d-cb5dfbc4239d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:55.716599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1628fa-81b0-4255-ab2d-cb5dfbc4239d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:55.716875 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1628fa-81b0-4255-ab2d-cb5dfbc4239d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:55.717141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1628fa-81b0-4255-ab2d-cb5dfbc4239d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:55.730877 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1628fa-81b0-4255-ab2d-cb5dfbc4239d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0a1628fa-81b0-4255-ab2d-cb5dfbc4239d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:53:55.736883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:53:55.742061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 37/73] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:53:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:53:55.744282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1628fa-81b0-4255-ab2d-cb5dfbc4239d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:55.769738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86511cbc-be4e-452e-bda9-987ce431231c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:55.826950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bec0fe-f2a1-4ba4-9fa8-23f6a1299c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:55.827952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bec0fe-f2a1-4ba4-9fa8-23f6a1299c2e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:55.829945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bec0fe-f2a1-4ba4-9fa8-23f6a1299c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:55.830409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bec0fe-f2a1-4ba4-9fa8-23f6a1299c2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:55.850914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bec0fe-f2a1-4ba4-9fa8-23f6a1299c2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b8bec0fe-f2a1-4ba4-9fa8-23f6a1299c2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:53:55.860711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:53:55.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:53:55.862591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:53:55.866124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 37/74] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:53:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:53:55.867679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bec0fe-f2a1-4ba4-9fa8-23f6a1299c2e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:55.878658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:55.908682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1628fa-81b0-4255-ab2d-cb5dfbc4239d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:55.949133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb743d3-63cd-467f-a7c6-d17ac3d5f0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:55.949740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb743d3-63cd-467f-a7c6-d17ac3d5f0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:55.950239 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb743d3-63cd-467f-a7c6-d17ac3d5f0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:55.950606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb743d3-63cd-467f-a7c6-d17ac3d5f0c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:55.951588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb743d3-63cd-467f-a7c6-d17ac3d5f0c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:55.951993 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb743d3-63cd-467f-a7c6-d17ac3d5f0c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:55.954591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb743d3-63cd-467f-a7c6-d17ac3d5f0c5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:55.970766 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 38/75] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:53:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:55.971672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb743d3-63cd-467f-a7c6-d17ac3d5f0c5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:55.994700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bec0fe-f2a1-4ba4-9fa8-23f6a1299c2e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:56.035945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:56.049988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984dc695-d814-4d59-a92e-ee8d1955179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:56.050774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dc695-d814-4d59-a92e-ee8d1955179a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:56.051214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dc695-d814-4d59-a92e-ee8d1955179a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:56.051670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984dc695-d814-4d59-a92e-ee8d1955179a None admin] PATH_INFO: `/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:56.068851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984dc695-d814-4d59-a92e-ee8d1955179a None admin] RBAC: Authorizing `identity:get_user(user_id=f11433fdb39c41a9b2b8dd15ec05cb21)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:56.070753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984dc695-d814-4d59-a92e-ee8d1955179a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:56.072853 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 38/76] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:53:55 2022] GET /identity/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:56.073994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22922e8b-8acb-4cfa-8569-78b4ba433faa None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:56.074261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22922e8b-8acb-4cfa-8569-78b4ba433faa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:56.074539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22922e8b-8acb-4cfa-8569-78b4ba433faa None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:56.075916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 39/77] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:53:56.079205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22922e8b-8acb-4cfa-8569-78b4ba433faa None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:56.138898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb743d3-63cd-467f-a7c6-d17ac3d5f0c5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:56.177124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cea91a-5262-4cec-9a8c-b15eb2aa5562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:56.177754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cea91a-5262-4cec-9a8c-b15eb2aa5562 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:56.178068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cea91a-5262-4cec-9a8c-b15eb2aa5562 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:56.178346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cea91a-5262-4cec-9a8c-b15eb2aa5562 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:56.179044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cea91a-5262-4cec-9a8c-b15eb2aa5562 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:56.179351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cea91a-5262-4cec-9a8c-b15eb2aa5562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:56.180807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cea91a-5262-4cec-9a8c-b15eb2aa5562 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:56.186755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:56.192086 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 39/78] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:53:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:56.197805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cea91a-5262-4cec-9a8c-b15eb2aa5562 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:56.235532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22922e8b-8acb-4cfa-8569-78b4ba433faa None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:56.278313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106363c-0f56-4ded-a8ae-2d1cb1840b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:56.278933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106363c-0f56-4ded-a8ae-2d1cb1840b5f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:56.279214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106363c-0f56-4ded-a8ae-2d1cb1840b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:56.279494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c106363c-0f56-4ded-a8ae-2d1cb1840b5f None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:56.280185 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106363c-0f56-4ded-a8ae-2d1cb1840b5f None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:56.280494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106363c-0f56-4ded-a8ae-2d1cb1840b5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:56.282027 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c106363c-0f56-4ded-a8ae-2d1cb1840b5f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:56.298299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 40/79] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:53:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:56.300050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106363c-0f56-4ded-a8ae-2d1cb1840b5f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:56.329918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cea91a-5262-4cec-9a8c-b15eb2aa5562 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:56.355474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:56.386561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f666f96-0b06-4b9f-a6d0-3a829505ebef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:56.387301 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f666f96-0b06-4b9f-a6d0-3a829505ebef None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:56.387668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f666f96-0b06-4b9f-a6d0-3a829505ebef None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:56.388028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f666f96-0b06-4b9f-a6d0-3a829505ebef None admin] PATH_INFO: `/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:56.389615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f666f96-0b06-4b9f-a6d0-3a829505ebef None admin] RBAC: Authorizing `identity:get_user(user_id=f11433fdb39c41a9b2b8dd15ec05cb21)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:56.391812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f666f96-0b06-4b9f-a6d0-3a829505ebef None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:56.393969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 40/80] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:53:56 2022] GET /identity/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:56.395669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0077ef-7a94-480b-9e9d-4f32a44c634d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:56.396078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0077ef-7a94-480b-9e9d-4f32a44c634d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:56.396429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0077ef-7a94-480b-9e9d-4f32a44c634d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:56.417668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4f0077ef-7a94-480b-9e9d-4f32a44c634d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:53:56.419073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4f0077ef-7a94-480b-9e9d-4f32a44c634d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:53:56.423824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c106363c-0f56-4ded-a8ae-2d1cb1840b5f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:56.472074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bc041-d53e-460d-90a8-8adc5a899b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:56.473086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bc041-d53e-460d-90a8-8adc5a899b6b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:56.473552 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bc041-d53e-460d-90a8-8adc5a899b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:56.474058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bc041-d53e-460d-90a8-8adc5a899b6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:56.492482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:56.499401 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 41/81] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:53:56.501381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0077ef-7a94-480b-9e9d-4f32a44c634d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:56.592609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bc041-d53e-460d-90a8-8adc5a899b6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:56.595297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bc041-d53e-460d-90a8-8adc5a899b6b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:56.659897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0077ef-7a94-480b-9e9d-4f32a44c634d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:56.679315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:56.725807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffc3bc-f04d-4cf5-96b4-cd9bef02ee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:56.726854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffc3bc-f04d-4cf5-96b4-cd9bef02ee9e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:56.727313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffc3bc-f04d-4cf5-96b4-cd9bef02ee9e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:56.727767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffc3bc-f04d-4cf5-96b4-cd9bef02ee9e None admin] PATH_INFO: `/v3/users/3476c91ad03941919217e0940471d201` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:56.729801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ffc3bc-f04d-4cf5-96b4-cd9bef02ee9e None admin] RBAC: Authorizing `identity:get_user(user_id=3476c91ad03941919217e0940471d201)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:56.732501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ffc3bc-f04d-4cf5-96b4-cd9bef02ee9e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:56.735544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 42/82] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:53:56 2022] GET /identity/v3/users/3476c91ad03941919217e0940471d201 => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:56.736922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffc3bc-f04d-4cf5-96b4-cd9bef02ee9e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:56.762199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 41/83] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:53:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:53:56.763235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bc041-d53e-460d-90a8-8adc5a899b6b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:56.840399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:56.893097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffc3bc-f04d-4cf5-96b4-cd9bef02ee9e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:56.904264 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bc041-d53e-460d-90a8-8adc5a899b6b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:56.938432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e395b3-08d6-4b6f-bdc4-f144585763be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:56.939084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e395b3-08d6-4b6f-bdc4-f144585763be None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:56.939366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e395b3-08d6-4b6f-bdc4-f144585763be None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:56.939645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e395b3-08d6-4b6f-bdc4-f144585763be None admin] PATH_INFO: `/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:56.940820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e395b3-08d6-4b6f-bdc4-f144585763be None admin] RBAC: Authorizing `identity:get_user(user_id=f11433fdb39c41a9b2b8dd15ec05cb21)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:56.942348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e395b3-08d6-4b6f-bdc4-f144585763be None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:56.943977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 43/84] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:53:56 2022] GET /identity/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:56.944756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a16e3-5e99-4e4c-a186-9fa0c805076d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:56.944974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e395b3-08d6-4b6f-bdc4-f144585763be None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:56.945849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a16e3-5e99-4e4c-a186-9fa0c805076d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:56.946356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a16e3-5e99-4e4c-a186-9fa0c805076d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:56.946860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144a16e3-5e99-4e4c-a186-9fa0c805076d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:56.972794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:57.057946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a16e3-5e99-4e4c-a186-9fa0c805076d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:57.059646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144a16e3-5e99-4e4c-a186-9fa0c805076d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:57.083602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e395b3-08d6-4b6f-bdc4-f144585763be None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:57.125294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ee7b9-f5d5-429f-85b0-52e51f608989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:57.126219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ee7b9-f5d5-429f-85b0-52e51f608989 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:57.126593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ee7b9-f5d5-429f-85b0-52e51f608989 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:57.126960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ee7b9-f5d5-429f-85b0-52e51f608989 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:57.131275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:57.226662 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 42/85] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:53:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:53:57.230925 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a16e3-5e99-4e4c-a186-9fa0c805076d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:57.233728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ee7b9-f5d5-429f-85b0-52e51f608989 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:57.236266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ee7b9-f5d5-429f-85b0-52e51f608989 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:57.291640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:57.376899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144a16e3-5e99-4e4c-a186-9fa0c805076d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:57.377342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 44/86] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:53:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:53:57.378906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ee7b9-f5d5-429f-85b0-52e51f608989 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:57.430014 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:57.432449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c404c4-d2fd-4611-b5cb-8f58af7e52bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:57.434012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c404c4-d2fd-4611-b5cb-8f58af7e52bd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:57.434481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c404c4-d2fd-4611-b5cb-8f58af7e52bd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:57.434907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c404c4-d2fd-4611-b5cb-8f58af7e52bd None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:57.436869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c404c4-d2fd-4611-b5cb-8f58af7e52bd None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:57.439967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c404c4-d2fd-4611-b5cb-8f58af7e52bd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:57.448413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 43/87] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:53:57 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:57.448413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c404c4-d2fd-4611-b5cb-8f58af7e52bd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:57.534463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ee7b9-f5d5-429f-85b0-52e51f608989 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:57.584066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c404c4-d2fd-4611-b5cb-8f58af7e52bd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:57.584428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c5670-90c0-460e-b393-02807376c9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:57.585179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5670-90c0-460e-b393-02807376c9ad None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:57.585629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5670-90c0-460e-b393-02807376c9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:57.586049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5670-90c0-460e-b393-02807376c9ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:57.587453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c5670-90c0-460e-b393-02807376c9ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:57.589064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c5670-90c0-460e-b393-02807376c9ad None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:57.590900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 45/88] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:53:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:57.592255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c5670-90c0-460e-b393-02807376c9ad None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:57.600612 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:57.627533 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed2b59-a165-4861-a7ac-4ad6cbb8e51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:57.628225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed2b59-a165-4861-a7ac-4ad6cbb8e51d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:57.628682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed2b59-a165-4861-a7ac-4ad6cbb8e51d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:57.629030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed2b59-a165-4861-a7ac-4ad6cbb8e51d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:57.715405 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed2b59-a165-4861-a7ac-4ad6cbb8e51d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:57.718055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed2b59-a165-4861-a7ac-4ad6cbb8e51d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:57.723435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c5670-90c0-460e-b393-02807376c9ad None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:57.725943 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:57.771907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c817796-5153-4c83-9aa3-40b468e026a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:57.772698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817796-5153-4c83-9aa3-40b468e026a2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:57.773103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817796-5153-4c83-9aa3-40b468e026a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:57.773513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817796-5153-4c83-9aa3-40b468e026a2 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:57.775085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c817796-5153-4c83-9aa3-40b468e026a2 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:57.776785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c817796-5153-4c83-9aa3-40b468e026a2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:57.778795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 46/89] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:53:57 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:57.779953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c817796-5153-4c83-9aa3-40b468e026a2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:57.859514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 44/90] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:53:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:53:57.860685 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed2b59-a165-4861-a7ac-4ad6cbb8e51d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:57.885198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:57.914896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c817796-5153-4c83-9aa3-40b468e026a2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:57.953169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da1b8a-52f5-4026-bf21-51e5c38ebb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:57.954381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da1b8a-52f5-4026-bf21-51e5c38ebb70 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:57.954658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da1b8a-52f5-4026-bf21-51e5c38ebb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:57.954937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da1b8a-52f5-4026-bf21-51e5c38ebb70 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:57.956158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da1b8a-52f5-4026-bf21-51e5c38ebb70 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:57.957637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da1b8a-52f5-4026-bf21-51e5c38ebb70 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:57.959573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 47/91] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:53:57 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:57.960160 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da1b8a-52f5-4026-bf21-51e5c38ebb70 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:58.017261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:58.032315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed2b59-a165-4861-a7ac-4ad6cbb8e51d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:58.078951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe283ee-bdea-4039-90d7-2c192ea50813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:58.079707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe283ee-bdea-4039-90d7-2c192ea50813 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:58.080056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe283ee-bdea-4039-90d7-2c192ea50813 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:58.080402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe283ee-bdea-4039-90d7-2c192ea50813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:58.081231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe283ee-bdea-4039-90d7-2c192ea50813 None admin] RBAC: Adding query filter params (role.id=f8c927860d6941559283c83fcaaf93d8, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15, user.id=f11433fdb39c41a9b2b8dd15ec05cb21) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:58.081606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe283ee-bdea-4039-90d7-2c192ea50813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:58.083523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe283ee-bdea-4039-90d7-2c192ea50813 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:58.093555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 45/92] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:53:57 2022] GET /identity/v3/role_assignments?user.id=f11433fdb39c41a9b2b8dd15ec05cb21&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:58.094572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe283ee-bdea-4039-90d7-2c192ea50813 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:58.102138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da1b8a-52f5-4026-bf21-51e5c38ebb70 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:58.158134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0b841-f76d-4b40-ab37-5c9903d677cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:58.159008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0b841-f76d-4b40-ab37-5c9903d677cc None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:58.159363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0b841-f76d-4b40-ab37-5c9903d677cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:58.160091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0b841-f76d-4b40-ab37-5c9903d677cc None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:58.161308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0b841-f76d-4b40-ab37-5c9903d677cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:58.163298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0b841-f76d-4b40-ab37-5c9903d677cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:58.208421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 48/93] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:53:57 2022] POST /identity/v3/users => generated 289 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:53:58.209028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0b841-f76d-4b40-ab37-5c9903d677cc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:58.286199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:58.344541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe283ee-bdea-4039-90d7-2c192ea50813 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:58.368540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:53:58.375967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0b841-f76d-4b40-ab37-5c9903d677cc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:58.418256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da945d2-dcc1-4ab4-a3cf-450ba93a00ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:58.418822 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da945d2-dcc1-4ab4-a3cf-450ba93a00ad None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:58.419104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da945d2-dcc1-4ab4-a3cf-450ba93a00ad None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:58.419383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da945d2-dcc1-4ab4-a3cf-450ba93a00ad None admin] PATH_INFO: `/v3/projects/0662068ace044ea9bd8c053b6a53291d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:58.420924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da945d2-dcc1-4ab4-a3cf-450ba93a00ad None admin] RBAC: Authorizing `identity:get_project(project_id=0662068ace044ea9bd8c053b6a53291d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:58.423315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da945d2-dcc1-4ab4-a3cf-450ba93a00ad None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:58.425420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 49/94] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:53:58 2022] GET /identity/v3/projects/0662068ace044ea9bd8c053b6a53291d => generated 312 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:58.426101 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da945d2-dcc1-4ab4-a3cf-450ba93a00ad None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:58.434522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d114e-cec2-4c91-9076-2936a413905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:58.435359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d114e-cec2-4c91-9076-2936a413905d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:58.435848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d114e-cec2-4c91-9076-2936a413905d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:58.436284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d114e-cec2-4c91-9076-2936a413905d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:58.438712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d114e-cec2-4c91-9076-2936a413905d None admin] RBAC: Adding query filter params (role.id=a631c57af3494b9c955799e0488518ae, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15, user.id=f11433fdb39c41a9b2b8dd15ec05cb21) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:58.438856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d114e-cec2-4c91-9076-2936a413905d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:58.442252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d114e-cec2-4c91-9076-2936a413905d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:58.452419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 46/95] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:53:58 2022] GET /identity/v3/role_assignments?user.id=f11433fdb39c41a9b2b8dd15ec05cb21&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=a631c57af3494b9c955799e0488518ae => generated 259 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:58.455129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d114e-cec2-4c91-9076-2936a413905d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:53:58.555903 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da945d2-dcc1-4ab4-a3cf-450ba93a00ad None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:58.602496 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdccd1-1075-43e7-8f04-f483dc8852ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:58.603167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdccd1-1075-43e7-8f04-f483dc8852ac None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:58.603441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdccd1-1075-43e7-8f04-f483dc8852ac None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:58.603867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdccd1-1075-43e7-8f04-f483dc8852ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:58.604784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdccd1-1075-43e7-8f04-f483dc8852ac None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15, user.id=3476c91ad03941919217e0940471d201) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:58.605102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdccd1-1075-43e7-8f04-f483dc8852ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:58.606642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdccd1-1075-43e7-8f04-f483dc8852ac None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:58.619732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 50/96] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:53:58 2022] GET /identity/v3/role_assignments?user.id=3476c91ad03941919217e0940471d201&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=8b350845f331423cb0494e3449b58310 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:53:58.664967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d114e-cec2-4c91-9076-2936a413905d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:53:58.709276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c6fcc-2099-4783-8e5b-6c4e6bca9cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:53:58.709854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c6fcc-2099-4783-8e5b-6c4e6bca9cfd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:53:58.710146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c6fcc-2099-4783-8e5b-6c4e6bca9cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:53:58.710401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c6fcc-2099-4783-8e5b-6c4e6bca9cfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:53:58.711176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c6fcc-2099-4783-8e5b-6c4e6bca9cfd None admin] RBAC: Adding query filter params (role.id=f8c927860d6941559283c83fcaaf93d8, scope.project.id=0662068ace044ea9bd8c053b6a53291d, user.id=f11433fdb39c41a9b2b8dd15ec05cb21) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:53:58.711553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c6fcc-2099-4783-8e5b-6c4e6bca9cfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:53:58.713086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c6fcc-2099-4783-8e5b-6c4e6bca9cfd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:53:58.721116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 47/97] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:53:58 2022] GET /identity/v3/role_assignments?user.id=f11433fdb39c41a9b2b8dd15ec05cb21&scope.project.id=0662068ace044ea9bd8c053b6a53291d&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:01.443551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c011154-8115-46af-aa11-78ac3d914cf3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:01.444033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c011154-8115-46af-aa11-78ac3d914cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:01.444033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c011154-8115-46af-aa11-78ac3d914cf3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:01.445407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 51/98] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:01.451245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f591ef7-791a-4d97-822e-4e823a8e84f7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:01.451739 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f591ef7-791a-4d97-822e-4e823a8e84f7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:01.452001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f591ef7-791a-4d97-822e-4e823a8e84f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:01.461557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:01.475030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7f591ef7-791a-4d97-822e-4e823a8e84f7 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:01.476278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7f591ef7-791a-4d97-822e-4e823a8e84f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:01.575115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 48/99] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:01.807372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23acfa-0257-44fc-9975-b5c0c7e913f5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:01.807775 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23acfa-0257-44fc-9975-b5c0c7e913f5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:01.807974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23acfa-0257-44fc-9975-b5c0c7e913f5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:01.809560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 52/100] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:01.826362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d941e3-222e-492f-8c5d-f6402fca9351 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:01.826663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d941e3-222e-492f-8c5d-f6402fca9351 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:01.826936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d941e3-222e-492f-8c5d-f6402fca9351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:01.852265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-06d941e3-222e-492f-8c5d-f6402fca9351 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:01.853536 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-06d941e3-222e-492f-8c5d-f6402fca9351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:01.910999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c676e-67d0-4815-8eb1-f4e19af19219 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:01.911310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c676e-67d0-4815-8eb1-f4e19af19219 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:01.911558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c676e-67d0-4815-8eb1-f4e19af19219 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:01.913223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 53/101] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:01.929389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121b590-0b72-4474-bbfc-4bbf92570936 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:01.929690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121b590-0b72-4474-bbfc-4bbf92570936 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:01.929993 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f121b590-0b72-4474-bbfc-4bbf92570936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:01.957078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f121b590-0b72-4474-bbfc-4bbf92570936 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:01.958351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f121b590-0b72-4474-bbfc-4bbf92570936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:01.992844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 49/102] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:02.001975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:02.076988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 54/103] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:02.272678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d941e3-222e-492f-8c5d-f6402fca9351 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:02.383059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:02.393173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94b71-cfe7-4b02-968e-491ad385b671 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:02.393521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94b71-cfe7-4b02-968e-491ad385b671 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:02.393743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed94b71-cfe7-4b02-968e-491ad385b671 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:02.395377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 55/104] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:54:02.401952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3c511-da34-4495-a247-08d1484bd0d5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:02.402267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3c511-da34-4495-a247-08d1484bd0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:02.402517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3c511-da34-4495-a247-08d1484bd0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:02.428057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4df3c511-da34-4495-a247-08d1484bd0d5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:02.429331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4df3c511-da34-4495-a247-08d1484bd0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:02.461813 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d941e3-222e-492f-8c5d-f6402fca9351 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:02.535199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80942192-9dfc-4ea4-b3f2-3630d0cfa4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:02.536623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942192-9dfc-4ea4-b3f2-3630d0cfa4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:02.537314 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942192-9dfc-4ea4-b3f2-3630d0cfa4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:02.538027 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80942192-9dfc-4ea4-b3f2-3630d0cfa4f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:02.559703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80942192-9dfc-4ea4-b3f2-3630d0cfa4f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:02.565860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 56/105] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:02.567921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa4c71-9f95-45ce-ba23-11acc8a09899 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:02.568449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa4c71-9f95-45ce-ba23-11acc8a09899 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:02.568931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa4c71-9f95-45ce-ba23-11acc8a09899 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-80942192-9dfc-4ea4-b3f2-3630d0cfa4f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:02.570217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:02.571104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:02.571881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:02.572487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 50/106] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:02.576588 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80942192-9dfc-4ea4-b3f2-3630d0cfa4f5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:02.576808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 57/107] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:02.581218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa4c71-9f95-45ce-ba23-11acc8a09899 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:02.717348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:02.739484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa4c71-9f95-45ce-ba23-11acc8a09899 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:02.772704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80942192-9dfc-4ea4-b3f2-3630d0cfa4f5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:02.801735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:02.806623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0572c-7665-4071-bf9f-b716909b4c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:02.808264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0572c-7665-4071-bf9f-b716909b4c71 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:02.809083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0572c-7665-4071-bf9f-b716909b4c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:02.809908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0572c-7665-4071-bf9f-b716909b4c71 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:02.811502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0572c-7665-4071-bf9f-b716909b4c71 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:02.812893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0572c-7665-4071-bf9f-b716909b4c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:02.816678 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0572c-7665-4071-bf9f-b716909b4c71 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:02.816869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618308ac-2c2d-4bf7-a9f2-ca6f80aed388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:02.817570 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618308ac-2c2d-4bf7-a9f2-ca6f80aed388 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:02.817971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618308ac-2c2d-4bf7-a9f2-ca6f80aed388 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:02.818406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618308ac-2c2d-4bf7-a9f2-ca6f80aed388 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:02.831716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618308ac-2c2d-4bf7-a9f2-ca6f80aed388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:02.835104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 58/108] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:02.838065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cecde-c0f7-43ff-b7c5-c1561088b6e4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:02.838907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cecde-c0f7-43ff-b7c5-c1561088b6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:02.839919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49cecde-c0f7-43ff-b7c5-c1561088b6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-618308ac-2c2d-4bf7-a9f2-ca6f80aed388 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:02.840042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:02.840860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:02.841595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 51/109] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:02.842341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618308ac-2c2d-4bf7-a9f2-ca6f80aed388 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:02.873674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e49cecde-c0f7-43ff-b7c5-c1561088b6e4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:02.875951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e49cecde-c0f7-43ff-b7c5-c1561088b6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:02.971872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618308ac-2c2d-4bf7-a9f2-ca6f80aed388 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:02.991462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 59/110] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:02.992349 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cecde-c0f7-43ff-b7c5-c1561088b6e4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:03.017462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603f586-9002-4f35-a40f-788d1aec7e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:03.018492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603f586-9002-4f35-a40f-788d1aec7e0f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:03.018792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603f586-9002-4f35-a40f-788d1aec7e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:03.019074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2603f586-9002-4f35-a40f-788d1aec7e0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:03.035908 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2603f586-9002-4f35-a40f-788d1aec7e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2603f586-9002-4f35-a40f-788d1aec7e0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:03.042419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:03.043385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:03.044113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 52/111] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:03.047251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603f586-9002-4f35-a40f-788d1aec7e0f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:03.106902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:03.119480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:03.141376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49cecde-c0f7-43ff-b7c5-c1561088b6e4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:03.193998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2603f586-9002-4f35-a40f-788d1aec7e0f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:03.208313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139496f-9e92-4f20-93e9-d469bd6b88e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:03.209421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139496f-9e92-4f20-93e9-d469bd6b88e3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:03.209931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139496f-9e92-4f20-93e9-d469bd6b88e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:03.210589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139496f-9e92-4f20-93e9-d469bd6b88e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:03.212759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139496f-9e92-4f20-93e9-d469bd6b88e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:03.215949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139496f-9e92-4f20-93e9-d469bd6b88e3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:03.220000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 60/112] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:54:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:03.221488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139496f-9e92-4f20-93e9-d469bd6b88e3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:03.241511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e273e-3881-4aaf-a78b-5c42544a2b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:03.242131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e273e-3881-4aaf-a78b-5c42544a2b09 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:03.242418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e273e-3881-4aaf-a78b-5c42544a2b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:03.242721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e273e-3881-4aaf-a78b-5c42544a2b09 None admin] PATH_INFO: `/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:03.244245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e273e-3881-4aaf-a78b-5c42544a2b09 None admin] RBAC: Authorizing `identity:get_user(user_id=f11433fdb39c41a9b2b8dd15ec05cb21)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:03.245701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e273e-3881-4aaf-a78b-5c42544a2b09 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:03.255193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 53/113] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:03 2022] GET /identity/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:03.255193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05808091-62be-4640-aaa1-c5fbc46a96ec None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:03.255193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05808091-62be-4640-aaa1-c5fbc46a96ec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:03.255193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05808091-62be-4640-aaa1-c5fbc46a96ec None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:03.255193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 54/114] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:03.255193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05808091-62be-4640-aaa1-c5fbc46a96ec None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:03.393977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05808091-62be-4640-aaa1-c5fbc46a96ec None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:03.404352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139496f-9e92-4f20-93e9-d469bd6b88e3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:03.409171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:03.447855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef60bc4-22c3-4853-8c1f-57dc15739b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:03.448142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d34881-dda6-4086-857b-f6ce1d8838db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:03.448610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d34881-dda6-4086-857b-f6ce1d8838db None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:03.448891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d34881-dda6-4086-857b-f6ce1d8838db None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:03.449026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef60bc4-22c3-4853-8c1f-57dc15739b14 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:03.449178 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d34881-dda6-4086-857b-f6ce1d8838db None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:03.449460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef60bc4-22c3-4853-8c1f-57dc15739b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:03.449860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d34881-dda6-4086-857b-f6ce1d8838db None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:03.449988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef60bc4-22c3-4853-8c1f-57dc15739b14 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:03.450167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d34881-dda6-4086-857b-f6ce1d8838db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:03.451231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef60bc4-22c3-4853-8c1f-57dc15739b14 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:03.451550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d34881-dda6-4086-857b-f6ce1d8838db None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:03.451760 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef60bc4-22c3-4853-8c1f-57dc15739b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:03.454874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef60bc4-22c3-4853-8c1f-57dc15739b14 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:03.460830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 61/115] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:03.461938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d34881-dda6-4086-857b-f6ce1d8838db None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:03.476107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 55/116] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:03.476107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef60bc4-22c3-4853-8c1f-57dc15739b14 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:03.478767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:03.594533 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef60bc4-22c3-4853-8c1f-57dc15739b14 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:03.611396 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d34881-dda6-4086-857b-f6ce1d8838db None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:03.632734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bdf2c-8b02-4614-b191-16106cc1a57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:03.633380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bdf2c-8b02-4614-b191-16106cc1a57d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:03.633684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bdf2c-8b02-4614-b191-16106cc1a57d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:03.633988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bdf2c-8b02-4614-b191-16106cc1a57d None admin] PATH_INFO: `/v3/projects/0662068ace044ea9bd8c053b6a53291d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:03.635205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bdf2c-8b02-4614-b191-16106cc1a57d None admin] RBAC: Authorizing `identity:get_project(project_id=0662068ace044ea9bd8c053b6a53291d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:03.636680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bdf2c-8b02-4614-b191-16106cc1a57d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:03.638427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 56/117] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:03 2022] GET /identity/v3/projects/0662068ace044ea9bd8c053b6a53291d => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:03.639719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342a9bb-9898-457d-93c5-298fa7b083ff None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:03.640141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342a9bb-9898-457d-93c5-298fa7b083ff None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:03.640456 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342a9bb-9898-457d-93c5-298fa7b083ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:03.651995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31068360-3325-4caa-ac66-980ee1a671af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:03.652809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31068360-3325-4caa-ac66-980ee1a671af None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:03.653092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31068360-3325-4caa-ac66-980ee1a671af None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:03.653463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31068360-3325-4caa-ac66-980ee1a671af None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:03.654388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31068360-3325-4caa-ac66-980ee1a671af None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:03.657386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31068360-3325-4caa-ac66-980ee1a671af None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:03.663138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b342a9bb-9898-457d-93c5-298fa7b083ff None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:03.664384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b342a9bb-9898-457d-93c5-298fa7b083ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:03.690842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 62/118] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:54:03 2022] POST /identity/v3/projects => generated 302 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:54:03.691797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31068360-3325-4caa-ac66-980ee1a671af None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:03.715106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:03.767575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 57/119] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:03.778159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:03.778837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342a9bb-9898-457d-93c5-298fa7b083ff None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:03.870929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31068360-3325-4caa-ac66-980ee1a671af None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:03.916639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181bb37-906a-41fc-953a-fffb68e34f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:03.917298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181bb37-906a-41fc-953a-fffb68e34f8c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:03.917582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181bb37-906a-41fc-953a-fffb68e34f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:03.917890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181bb37-906a-41fc-953a-fffb68e34f8c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:03.933243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181bb37-906a-41fc-953a-fffb68e34f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8181bb37-906a-41fc-953a-fffb68e34f8c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:03.938951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:03.940112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:03.941163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 63/120] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:54:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:03.941819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342a9bb-9898-457d-93c5-298fa7b083ff None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:03.941891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181bb37-906a-41fc-953a-fffb68e34f8c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:04.001355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44626a8d-3faf-427f-9d08-52fc48620a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:04.002691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44626a8d-3faf-427f-9d08-52fc48620a59 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:04.003409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44626a8d-3faf-427f-9d08-52fc48620a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:04.004047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44626a8d-3faf-427f-9d08-52fc48620a59 None admin] PATH_INFO: `/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:04.007129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44626a8d-3faf-427f-9d08-52fc48620a59 None admin] RBAC: Authorizing `identity:get_user(user_id=f11433fdb39c41a9b2b8dd15ec05cb21)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:04.012081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44626a8d-3faf-427f-9d08-52fc48620a59 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:04.017333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 58/121] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:03 2022] GET /identity/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21 => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:04.022642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44626a8d-3faf-427f-9d08-52fc48620a59 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:04.085528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:04.114127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181bb37-906a-41fc-953a-fffb68e34f8c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:04.140261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:04.160158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6adc29-1c63-47b4-a766-f06436bffce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:04.160977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6adc29-1c63-47b4-a766-f06436bffce8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:04.161389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6adc29-1c63-47b4-a766-f06436bffce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:04.161550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44626a8d-3faf-427f-9d08-52fc48620a59 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:04.161893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6adc29-1c63-47b4-a766-f06436bffce8 None admin] PATH_INFO: `/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:04.163430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6adc29-1c63-47b4-a766-f06436bffce8 None admin] RBAC: Authorizing `identity:get_user(user_id=f11433fdb39c41a9b2b8dd15ec05cb21)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:04.165669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6adc29-1c63-47b4-a766-f06436bffce8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:04.173367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 64/122] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:03 2022] GET /identity/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:04.173367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6adc29-1c63-47b4-a766-f06436bffce8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:04.214424 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5fd24-7bfb-445b-ad39-c0732e74315e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:04.215436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5fd24-7bfb-445b-ad39-c0732e74315e None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:04.215709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5fd24-7bfb-445b-ad39-c0732e74315e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:04.216298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b5fd24-7bfb-445b-ad39-c0732e74315e None admin] PATH_INFO: `/v3/projects/0662068ace044ea9bd8c053b6a53291d/users/f11433fdb39c41a9b2b8dd15ec05cb21/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:04.219705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5fd24-7bfb-445b-ad39-c0732e74315e None admin] RBAC: Authorizing `identity:create_grant(project_id=0662068ace044ea9bd8c053b6a53291d, user_id=f11433fdb39c41a9b2b8dd15ec05cb21, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:04.222411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b5fd24-7bfb-445b-ad39-c0732e74315e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:04.244657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 59/123] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:54:04 2022] PUT /identity/v3/projects/0662068ace044ea9bd8c053b6a53291d/users/f11433fdb39c41a9b2b8dd15ec05cb21/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:04.245988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5fd24-7bfb-445b-ad39-c0732e74315e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:04.380826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6adc29-1c63-47b4-a766-f06436bffce8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:04.451349 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbd839-3d02-48eb-bd7d-66e450fe1d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:04.451895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd839-3d02-48eb-bd7d-66e450fe1d8d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:04.452184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd839-3d02-48eb-bd7d-66e450fe1d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:04.452449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbd839-3d02-48eb-bd7d-66e450fe1d8d None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:04.453704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbd839-3d02-48eb-bd7d-66e450fe1d8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:04.454275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbd839-3d02-48eb-bd7d-66e450fe1d8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:04.455897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbd839-3d02-48eb-bd7d-66e450fe1d8d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:04.465844 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 65/124] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:04.466735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbd839-3d02-48eb-bd7d-66e450fe1d8d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:04.474616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:04.480520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b5fd24-7bfb-445b-ad39-c0732e74315e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:04.528445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:04.546288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470647ea-73b5-4837-a291-cbc5e6a84a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:04.547450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470647ea-73b5-4837-a291-cbc5e6a84a8e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:04.547948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470647ea-73b5-4837-a291-cbc5e6a84a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:04.548466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470647ea-73b5-4837-a291-cbc5e6a84a8e None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:04.551018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470647ea-73b5-4837-a291-cbc5e6a84a8e None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:04.553858 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470647ea-73b5-4837-a291-cbc5e6a84a8e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:04.557118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 60/125] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:04 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:04.558880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470647ea-73b5-4837-a291-cbc5e6a84a8e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:04.589296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbd839-3d02-48eb-bd7d-66e450fe1d8d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:04.630324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55693a4-e7b6-4dd1-a672-87cbe137d11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:04.631435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55693a4-e7b6-4dd1-a672-87cbe137d11d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:04.631435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55693a4-e7b6-4dd1-a672-87cbe137d11d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:04.632032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55693a4-e7b6-4dd1-a672-87cbe137d11d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:04.644170 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55693a4-e7b6-4dd1-a672-87cbe137d11d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b55693a4-e7b6-4dd1-a672-87cbe137d11d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:04.649631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:04.652799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:04.655854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 66/126] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:04.664308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55693a4-e7b6-4dd1-a672-87cbe137d11d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:04.762116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470647ea-73b5-4837-a291-cbc5e6a84a8e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:04.773322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55693a4-e7b6-4dd1-a672-87cbe137d11d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:04.786888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:04.816545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d43bce-068e-42ed-acaf-1412a40730f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:04.817922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43bce-068e-42ed-acaf-1412a40730f8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:04.818371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43bce-068e-42ed-acaf-1412a40730f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:04.819052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d43bce-068e-42ed-acaf-1412a40730f8 None admin] PATH_INFO: `/v3/users/3476c91ad03941919217e0940471d201` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:04.820462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d43bce-068e-42ed-acaf-1412a40730f8 None admin] RBAC: Authorizing `identity:get_user(user_id=3476c91ad03941919217e0940471d201)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:04.822396 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d43bce-068e-42ed-acaf-1412a40730f8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:04.824916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 67/127] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:04 2022] GET /identity/v3/users/3476c91ad03941919217e0940471d201 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:04.825921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d43bce-068e-42ed-acaf-1412a40730f8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:04.838465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfce667-1c5d-4998-9cdf-eda73f91483f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:04.839524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce667-1c5d-4998-9cdf-eda73f91483f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:04.840007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce667-1c5d-4998-9cdf-eda73f91483f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:04.840504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfce667-1c5d-4998-9cdf-eda73f91483f None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:04.842600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfce667-1c5d-4998-9cdf-eda73f91483f None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:04.845745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfce667-1c5d-4998-9cdf-eda73f91483f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:04.849363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 61/128] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:04 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:04.850962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfce667-1c5d-4998-9cdf-eda73f91483f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:04.916418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:04.969813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d43bce-068e-42ed-acaf-1412a40730f8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:05.024613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfce667-1c5d-4998-9cdf-eda73f91483f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:05.037169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321c41a-06c0-4a9a-b76c-9aa9752f3892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:05.038379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321c41a-06c0-4a9a-b76c-9aa9752f3892 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:05.039039 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321c41a-06c0-4a9a-b76c-9aa9752f3892 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:05.039558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321c41a-06c0-4a9a-b76c-9aa9752f3892 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/users/f11433fdb39c41a9b2b8dd15ec05cb21/roles/a631c57af3494b9c955799e0488518ae` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:05.043069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321c41a-06c0-4a9a-b76c-9aa9752f3892 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd6e674b6d240749abdcbbd0b748d15, user_id=f11433fdb39c41a9b2b8dd15ec05cb21, role_id=a631c57af3494b9c955799e0488518ae)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:05.045193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321c41a-06c0-4a9a-b76c-9aa9752f3892 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:05.072040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72c368-4c43-4644-a64e-e23567946d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:05.072657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72c368-4c43-4644-a64e-e23567946d44 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:05.072945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72c368-4c43-4644-a64e-e23567946d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:05.073225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72c368-4c43-4644-a64e-e23567946d44 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:05.074100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72c368-4c43-4644-a64e-e23567946d44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:05.074528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72c368-4c43-4644-a64e-e23567946d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:05.075099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 68/129] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:54:04 2022] PUT /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/users/f11433fdb39c41a9b2b8dd15ec05cb21/roles/a631c57af3494b9c955799e0488518ae => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:05.075953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d72c368-4c43-4644-a64e-e23567946d44 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:05.076842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321c41a-06c0-4a9a-b76c-9aa9752f3892 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:05.104143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 62/130] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:05.105030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72c368-4c43-4644-a64e-e23567946d44 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:05.157547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:05.294630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321c41a-06c0-4a9a-b76c-9aa9752f3892 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:05.326006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:05.331915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d72c368-4c43-4644-a64e-e23567946d44 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:05.353360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dc209-91e3-43d2-bb56-1f0092742c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:05.355007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dc209-91e3-43d2-bb56-1f0092742c76 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:05.355544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dc209-91e3-43d2-bb56-1f0092742c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:05.356416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dc209-91e3-43d2-bb56-1f0092742c76 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:05.359368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02dc209-91e3-43d2-bb56-1f0092742c76 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:05.363888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02dc209-91e3-43d2-bb56-1f0092742c76 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:05.370009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 69/131] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:05 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:05.371663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dc209-91e3-43d2-bb56-1f0092742c76 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:05.384959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bda1fb-5153-400a-b4e5-cab4a1a84562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:05.385651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bda1fb-5153-400a-b4e5-cab4a1a84562 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:05.386002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bda1fb-5153-400a-b4e5-cab4a1a84562 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:05.386293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bda1fb-5153-400a-b4e5-cab4a1a84562 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/users/f11433fdb39c41a9b2b8dd15ec05cb21/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:05.389477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bda1fb-5153-400a-b4e5-cab4a1a84562 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd6e674b6d240749abdcbbd0b748d15, user_id=f11433fdb39c41a9b2b8dd15ec05cb21, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:05.391959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bda1fb-5153-400a-b4e5-cab4a1a84562 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:05.409012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 63/132] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:54:05 2022] PUT /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/users/f11433fdb39c41a9b2b8dd15ec05cb21/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:05.409988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bda1fb-5153-400a-b4e5-cab4a1a84562 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:05.572023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dc209-91e3-43d2-bb56-1f0092742c76 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:05.582620 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:05.596266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bda1fb-5153-400a-b4e5-cab4a1a84562 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:05.642602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ac359-08f5-45af-bf63-29cb251161a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:05.643385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ac359-08f5-45af-bf63-29cb251161a1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:05.643612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ac359-08f5-45af-bf63-29cb251161a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:05.643897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ac359-08f5-45af-bf63-29cb251161a1 None admin] PATH_INFO: `/v3/users/3c71bc88f3914d2bb5ae4e7c3fa1ee8f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:05.668007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb5238-6570-4164-8d8f-ab4d4b11594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:05.669532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb5238-6570-4164-8d8f-ab4d4b11594b None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:05.670278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb5238-6570-4164-8d8f-ab4d4b11594b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:05.670992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb5238-6570-4164-8d8f-ab4d4b11594b None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/users/3476c91ad03941919217e0940471d201/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:05.675254 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ac359-08f5-45af-bf63-29cb251161a1 None admin] RBAC: Authorizing `identity:get_user(user_id=3c71bc88f3914d2bb5ae4e7c3fa1ee8f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:05.675830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb5238-6570-4164-8d8f-ab4d4b11594b None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd6e674b6d240749abdcbbd0b748d15, user_id=3476c91ad03941919217e0940471d201, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:05.677042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ac359-08f5-45af-bf63-29cb251161a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:05.678908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 70/133] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:05 2022] GET /identity/v3/users/3c71bc88f3914d2bb5ae4e7c3fa1ee8f => generated 289 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:05.679363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb5238-6570-4164-8d8f-ab4d4b11594b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:05.684973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ac359-08f5-45af-bf63-29cb251161a1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:05.716118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 64/134] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:54:05 2022] PUT /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/users/3476c91ad03941919217e0940471d201/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:05.924701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ac359-08f5-45af-bf63-29cb251161a1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:06.001753 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089e9e6-adcf-46c8-8dc9-ceb618846bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:06.003291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e9e6-adcf-46c8-8dc9-ceb618846bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:06.004011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e9e6-adcf-46c8-8dc9-ceb618846bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:06.004716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089e9e6-adcf-46c8-8dc9-ceb618846bd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:06.094961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:06.196533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089e9e6-adcf-46c8-8dc9-ceb618846bd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:06.199468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089e9e6-adcf-46c8-8dc9-ceb618846bd4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:06.408211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 71/135] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 20:54:06.417057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb5238-6570-4164-8d8f-ab4d4b11594b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:06.445148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:06.559233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb5238-6570-4164-8d8f-ab4d4b11594b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:06.606508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924d4ef-1bd9-4a12-a384-6ff61b14d38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:06.607477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924d4ef-1bd9-4a12-a384-6ff61b14d38d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:06.607477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924d4ef-1bd9-4a12-a384-6ff61b14d38d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:06.607718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924d4ef-1bd9-4a12-a384-6ff61b14d38d None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:06.608908 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924d4ef-1bd9-4a12-a384-6ff61b14d38d None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:06.611122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924d4ef-1bd9-4a12-a384-6ff61b14d38d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:06.612859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 65/136] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:06 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:06.619385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089e9e6-adcf-46c8-8dc9-ceb618846bd4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:06.767171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:06.848075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089e9e6-adcf-46c8-8dc9-ceb618846bd4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:06.928383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1881b36-e53e-424f-9d12-d91064474899 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:06.928660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1881b36-e53e-424f-9d12-d91064474899 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:06.928971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1881b36-e53e-424f-9d12-d91064474899 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:06.929463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347bd63-d008-4d50-80ea-8df261654ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:06.930581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 66/137] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:06.931267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347bd63-d008-4d50-80ea-8df261654ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:06.933733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347bd63-d008-4d50-80ea-8df261654ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:06.934662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347bd63-d008-4d50-80ea-8df261654ed2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:06.936075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721b461-047a-447e-8e25-6f5d68904b15 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:06.936362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721b461-047a-447e-8e25-6f5d68904b15 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:06.936640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d721b461-047a-447e-8e25-6f5d68904b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:06.937340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347bd63-d008-4d50-80ea-8df261654ed2 None admin] RBAC: Adding query filter params (role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15, user.id=3c71bc88f3914d2bb5ae4e7c3fa1ee8f) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:06.938762 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347bd63-d008-4d50-80ea-8df261654ed2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:06.941835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347bd63-d008-4d50-80ea-8df261654ed2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:06.958721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 72/138] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:06 2022] GET /identity/v3/role_assignments?user.id=3c71bc88f3914d2bb5ae4e7c3fa1ee8f&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f => generated 259 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:06.978659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d721b461-047a-447e-8e25-6f5d68904b15 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:06.979906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d721b461-047a-447e-8e25-6f5d68904b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:07.144130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 67/139] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:07.684692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347bd63-d008-4d50-80ea-8df261654ed2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:07.935636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347bd63-d008-4d50-80ea-8df261654ed2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:07.980980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:07.996622 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f92f4-b5c7-45c2-b164-ae6e0cd62764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:07.998278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f92f4-b5c7-45c2-b164-ae6e0cd62764 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:07.999394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f92f4-b5c7-45c2-b164-ae6e0cd62764 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:08.000743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f92f4-b5c7-45c2-b164-ae6e0cd62764 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:08.003145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f92f4-b5c7-45c2-b164-ae6e0cd62764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:08.005120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f92f4-b5c7-45c2-b164-ae6e0cd62764 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:08.007367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 73/140] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:54:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:08.014042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721b461-047a-447e-8e25-6f5d68904b15 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:08.050208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:08.101890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef964a9-adf6-4a1d-b0eb-625d7f308677 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:08.103564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef964a9-adf6-4a1d-b0eb-625d7f308677 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:08.103564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef964a9-adf6-4a1d-b0eb-625d7f308677 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:08.104193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 74/141] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:54:08.111374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d637708-d060-4ba0-9535-4888afe7900a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:08.111660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d637708-d060-4ba0-9535-4888afe7900a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:08.111939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d637708-d060-4ba0-9535-4888afe7900a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:08.150872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6d637708-d060-4ba0-9535-4888afe7900a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:08.152135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6d637708-d060-4ba0-9535-4888afe7900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:08.212770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d721b461-047a-447e-8e25-6f5d68904b15 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:08.289360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 75/142] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:08.300604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78bac00-d67e-4d98-9dd0-13bb0fd78e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:08.301989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bac00-d67e-4d98-9dd0-13bb0fd78e7c None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:08.302652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bac00-d67e-4d98-9dd0-13bb0fd78e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:08.303312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78bac00-d67e-4d98-9dd0-13bb0fd78e7c None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:08.304699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bac00-d67e-4d98-9dd0-13bb0fd78e7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:08.307130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78bac00-d67e-4d98-9dd0-13bb0fd78e7c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:08.342145 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 68/143] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:54:08 2022] POST /identity/v3/users => generated 283 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:54:08.575286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d637708-d060-4ba0-9535-4888afe7900a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:08.764093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74355b-c120-45fd-8aa0-24ae0bda4179 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:08.764699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74355b-c120-45fd-8aa0-24ae0bda4179 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:08.765157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74355b-c120-45fd-8aa0-24ae0bda4179 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:08.771469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 69/144] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:08.774219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d637708-d060-4ba0-9535-4888afe7900a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:08.777668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6bb07-9ea9-4602-9f8d-bfe41cfd806b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:08.778288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6bb07-9ea9-4602-9f8d-bfe41cfd806b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:08.778894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6bb07-9ea9-4602-9f8d-bfe41cfd806b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:08.814389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-28f6bb07-9ea9-4602-9f8d-bfe41cfd806b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:08.815977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-28f6bb07-9ea9-4602-9f8d-bfe41cfd806b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:08.842252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:08.859518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f502f3-fdc9-4ce8-a520-b4188dc60ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:08.860184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f502f3-fdc9-4ce8-a520-b4188dc60ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:08.860467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f502f3-fdc9-4ce8-a520-b4188dc60ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:08.864163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f502f3-fdc9-4ce8-a520-b4188dc60ec2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:08.884077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f502f3-fdc9-4ce8-a520-b4188dc60ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-41f502f3-fdc9-4ce8-a520-b4188dc60ec2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:08.889053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:08.889973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:08.891264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 76/145] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:08.901365 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f502f3-fdc9-4ce8-a520-b4188dc60ec2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:08.981013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 70/146] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:09.094413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f502f3-fdc9-4ce8-a520-b4188dc60ec2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:09.149202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1f3b36-c827-4699-b4f9-0633d89e8637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:09.149862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f3b36-c827-4699-b4f9-0633d89e8637 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:09.150150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f3b36-c827-4699-b4f9-0633d89e8637 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:09.150426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1f3b36-c827-4699-b4f9-0633d89e8637 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:09.151133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f3b36-c827-4699-b4f9-0633d89e8637 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:09.151444 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f3b36-c827-4699-b4f9-0633d89e8637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:09.153014 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1f3b36-c827-4699-b4f9-0633d89e8637 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:09.167192 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 77/147] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:09.173743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6bb07-9ea9-4602-9f8d-bfe41cfd806b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:09.176532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:09.261512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc8f31-0cd9-4843-a29e-d4f0b2667221 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:09.261889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc8f31-0cd9-4843-a29e-d4f0b2667221 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:09.262106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc8f31-0cd9-4843-a29e-d4f0b2667221 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:09.264664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 78/148] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:09.271336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a76f-b6c6-43b2-b5bd-9b3d5c2dc276 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:09.271673 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a76f-b6c6-43b2-b5bd-9b3d5c2dc276 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:09.271926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a76f-b6c6-43b2-b5bd-9b3d5c2dc276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:09.301399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6bb07-9ea9-4602-9f8d-bfe41cfd806b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:09.310962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7391a76f-b6c6-43b2-b5bd-9b3d5c2dc276 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:09.312756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7391a76f-b6c6-43b2-b5bd-9b3d5c2dc276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:09.346508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb80ac7-963c-49a5-8028-13e35f2c791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:09.347092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb80ac7-963c-49a5-8028-13e35f2c791d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:09.347377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb80ac7-963c-49a5-8028-13e35f2c791d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:09.347659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb80ac7-963c-49a5-8028-13e35f2c791d None admin] PATH_INFO: `/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:09.348884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb80ac7-963c-49a5-8028-13e35f2c791d None admin] RBAC: Authorizing `identity:get_user(user_id=f11433fdb39c41a9b2b8dd15ec05cb21)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:09.351135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb80ac7-963c-49a5-8028-13e35f2c791d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:09.352961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 71/149] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:09 2022] GET /identity/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:09.359770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb80ac7-963c-49a5-8028-13e35f2c791d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:09.418361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:09.446860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:09.451564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 79/150] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:09.485843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb80ac7-963c-49a5-8028-13e35f2c791d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:09.527631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46231dfd-f892-48cc-902e-5e43301c25e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:09.528263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46231dfd-f892-48cc-902e-5e43301c25e0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:09.528543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46231dfd-f892-48cc-902e-5e43301c25e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:09.528819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46231dfd-f892-48cc-902e-5e43301c25e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:09.611413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46231dfd-f892-48cc-902e-5e43301c25e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:09.612874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46231dfd-f892-48cc-902e-5e43301c25e0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:09.697431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a76f-b6c6-43b2-b5bd-9b3d5c2dc276 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:09.706027 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:09.737965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 72/151] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 20:54:09.745100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46231dfd-f892-48cc-902e-5e43301c25e0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:09.875303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46231dfd-f892-48cc-902e-5e43301c25e0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:09.921315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edb72a-4906-4907-8e62-6e7f35b4026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:09.922620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edb72a-4906-4907-8e62-6e7f35b4026c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:09.922929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edb72a-4906-4907-8e62-6e7f35b4026c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:09.923249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edb72a-4906-4907-8e62-6e7f35b4026c None admin] PATH_INFO: `/v3/projects/0662068ace044ea9bd8c053b6a53291d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:09.924546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edb72a-4906-4907-8e62-6e7f35b4026c None admin] RBAC: Authorizing `identity:get_project(project_id=0662068ace044ea9bd8c053b6a53291d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:09.926134 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edb72a-4906-4907-8e62-6e7f35b4026c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:09.932557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 73/152] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:09 2022] GET /identity/v3/projects/0662068ace044ea9bd8c053b6a53291d => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:09.934369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edb72a-4906-4907-8e62-6e7f35b4026c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:09.965911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a76f-b6c6-43b2-b5bd-9b3d5c2dc276 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:09.983768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:10.051941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b629f-1915-45b5-b646-0464c1104b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:10.052856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b629f-1915-45b5-b646-0464c1104b6a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:10.053277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b629f-1915-45b5-b646-0464c1104b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:10.053723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b629f-1915-45b5-b646-0464c1104b6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:10.063140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edb72a-4906-4907-8e62-6e7f35b4026c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:10.089412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b629f-1915-45b5-b646-0464c1104b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ce0b629f-1915-45b5-b646-0464c1104b6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:10.103070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:10.105732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 80/153] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 408 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:10.107835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252ee9e-9b62-4131-8c14-6b019bb612c6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:10.112491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252ee9e-9b62-4131-8c14-6b019bb612c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:10.114348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a252ee9e-9b62-4131-8c14-6b019bb612c6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:10.117526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1994d8f3-64bd-4f1d-9941-d10d522ecf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:10.118452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 81/154] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:10 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:10.119021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994d8f3-64bd-4f1d-9941-d10d522ecf98 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:10.119684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994d8f3-64bd-4f1d-9941-d10d522ecf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:10.119856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252ee9e-9b62-4131-8c14-6b019bb612c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:10.120467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1994d8f3-64bd-4f1d-9941-d10d522ecf98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:10.121326 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994d8f3-64bd-4f1d-9941-d10d522ecf98 None admin] RBAC: Adding query filter params (role.id=f8c927860d6941559283c83fcaaf93d8, scope.project.id=0662068ace044ea9bd8c053b6a53291d, user.id=f11433fdb39c41a9b2b8dd15ec05cb21) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:10.121713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994d8f3-64bd-4f1d-9941-d10d522ecf98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:10.123460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1994d8f3-64bd-4f1d-9941-d10d522ecf98 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:10.135563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 74/155] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:09 2022] GET /identity/v3/role_assignments?user.id=f11433fdb39c41a9b2b8dd15ec05cb21&scope.project.id=0662068ace044ea9bd8c053b6a53291d&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:10.137078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887a010-2c05-43ef-a222-d09e5de9ea4b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:10.137369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887a010-2c05-43ef-a222-d09e5de9ea4b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:10.137653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5887a010-2c05-43ef-a222-d09e5de9ea4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:10.169595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5887a010-2c05-43ef-a222-d09e5de9ea4b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:10.170935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5887a010-2c05-43ef-a222-d09e5de9ea4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:10.263191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 75/156] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:10.264119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887a010-2c05-43ef-a222-d09e5de9ea4b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:10.293210 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:10.310443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:10.406589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5887a010-2c05-43ef-a222-d09e5de9ea4b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:10.467824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8388f55-a0fa-44cc-ac1e-6a6bdb3f9cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:10.468547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8388f55-a0fa-44cc-ac1e-6a6bdb3f9cec None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:10.468826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8388f55-a0fa-44cc-ac1e-6a6bdb3f9cec None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:10.469101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8388f55-a0fa-44cc-ac1e-6a6bdb3f9cec None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:10.469812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8388f55-a0fa-44cc-ac1e-6a6bdb3f9cec None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:10.470125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8388f55-a0fa-44cc-ac1e-6a6bdb3f9cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:10.472299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8388f55-a0fa-44cc-ac1e-6a6bdb3f9cec None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:10.489436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 76/157] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:10.496928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8388f55-a0fa-44cc-ac1e-6a6bdb3f9cec None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:10.576625 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a252ee9e-9b62-4131-8c14-6b019bb612c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:10.623920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:10.633995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8388f55-a0fa-44cc-ac1e-6a6bdb3f9cec None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:10.677629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55d13b-03f0-4c3a-9147-d95627795b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:10.678244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d13b-03f0-4c3a-9147-d95627795b5e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:10.678640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d13b-03f0-4c3a-9147-d95627795b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:10.678782 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d13b-03f0-4c3a-9147-d95627795b5e None admin] PATH_INFO: `/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:10.680254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55d13b-03f0-4c3a-9147-d95627795b5e None admin] RBAC: Authorizing `identity:get_user(user_id=f11433fdb39c41a9b2b8dd15ec05cb21)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:10.682016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b55d13b-03f0-4c3a-9147-d95627795b5e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:10.684235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 77/158] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:10 2022] GET /identity/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21 => generated 275 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:10.685407 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918545a6-f423-4e28-8c6a-6d822197ed8b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:10.685808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918545a6-f423-4e28-8c6a-6d822197ed8b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:10.686102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918545a6-f423-4e28-8c6a-6d822197ed8b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:10.688218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 78/159] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:10.693840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d655e20-25b6-4629-afc5-5acde70d7263 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:10.694177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d655e20-25b6-4629-afc5-5acde70d7263 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:10.694409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d655e20-25b6-4629-afc5-5acde70d7263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:10.702945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b923e-e227-448e-bf7d-04eec20587b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:10.704025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b923e-e227-448e-bf7d-04eec20587b4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:10.704514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b923e-e227-448e-bf7d-04eec20587b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:10.704994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b923e-e227-448e-bf7d-04eec20587b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:10.715503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5d655e20-25b6-4629-afc5-5acde70d7263 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:10.716743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5d655e20-25b6-4629-afc5-5acde70d7263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:10.730919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4b923e-e227-448e-bf7d-04eec20587b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ad4b923e-e227-448e-bf7d-04eec20587b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:10.749529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:10.751759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:10.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:10.758001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 82/160] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 639 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:10.759706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b923e-e227-448e-bf7d-04eec20587b4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:10.821811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 79/161] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:10.822725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d655e20-25b6-4629-afc5-5acde70d7263 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:10.978394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:10.984745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b923e-e227-448e-bf7d-04eec20587b4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:11.032468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d655e20-25b6-4629-afc5-5acde70d7263 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:11.052549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:11.072431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cde85-29ce-4650-82a3-fd1f37796ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:11.073510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cde85-29ce-4650-82a3-fd1f37796ebe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:11.074045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cde85-29ce-4650-82a3-fd1f37796ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:11.074550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0cde85-29ce-4650-82a3-fd1f37796ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:11.078693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b9617-7bf8-44ea-bc4d-3f243724b3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:11.079504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b9617-7bf8-44ea-bc4d-3f243724b3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:11.079815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b9617-7bf8-44ea-bc4d-3f243724b3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:11.080099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b9617-7bf8-44ea-bc4d-3f243724b3c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:11.080819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b9617-7bf8-44ea-bc4d-3f243724b3c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:11.081140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b9617-7bf8-44ea-bc4d-3f243724b3c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:11.082684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b9617-7bf8-44ea-bc4d-3f243724b3c8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:11.094323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 80/162] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:11.095059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b9617-7bf8-44ea-bc4d-3f243724b3c8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:11.269515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cde85-29ce-4650-82a3-fd1f37796ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:11.273114 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0cde85-29ce-4650-82a3-fd1f37796ebe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:11.328336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b9617-7bf8-44ea-bc4d-3f243724b3c8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:11.410864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501c7f6-cd54-4fc4-afd9-c12bad8ca19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:11.411553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501c7f6-cd54-4fc4-afd9-c12bad8ca19e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:11.411841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501c7f6-cd54-4fc4-afd9-c12bad8ca19e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:11.412149 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6501c7f6-cd54-4fc4-afd9-c12bad8ca19e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:11.437744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6501c7f6-cd54-4fc4-afd9-c12bad8ca19e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6501c7f6-cd54-4fc4-afd9-c12bad8ca19e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:11.447945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:11.449608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:11.451097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 81/163] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:54:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:11.455432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501c7f6-cd54-4fc4-afd9-c12bad8ca19e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:11.491419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:11.494174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:11.497875 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 83/164] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:11.499109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cde85-29ce-4650-82a3-fd1f37796ebe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:11.612709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0cde85-29ce-4650-82a3-fd1f37796ebe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:11.654345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380905fe-4fab-4502-a08c-fd0237183f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:11.655392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380905fe-4fab-4502-a08c-fd0237183f72 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:11.656428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380905fe-4fab-4502-a08c-fd0237183f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:11.656903 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380905fe-4fab-4502-a08c-fd0237183f72 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:11.670022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380905fe-4fab-4502-a08c-fd0237183f72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-380905fe-4fab-4502-a08c-fd0237183f72 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:11.677800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:11.678977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:11.679774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:11.682843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 84/165] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:11.684013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380905fe-4fab-4502-a08c-fd0237183f72 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:11.747691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6501c7f6-cd54-4fc4-afd9-c12bad8ca19e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:11.783155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:11.790392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a79ef-af47-4314-b3ee-54f65ef4e0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:11.792405 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a79ef-af47-4314-b3ee-54f65ef4e0ba None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:11.792787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a79ef-af47-4314-b3ee-54f65ef4e0ba None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:11.793492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a79ef-af47-4314-b3ee-54f65ef4e0ba None admin] PATH_INFO: `/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:11.796196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a79ef-af47-4314-b3ee-54f65ef4e0ba None admin] RBAC: Authorizing `identity:get_user(user_id=f11433fdb39c41a9b2b8dd15ec05cb21)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:11.797206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a79ef-af47-4314-b3ee-54f65ef4e0ba None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:11.799908 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 82/166] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:11 2022] GET /identity/v3/users/f11433fdb39c41a9b2b8dd15ec05cb21 => generated 275 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:11.801401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a79ef-af47-4314-b3ee-54f65ef4e0ba None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:11.881283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380905fe-4fab-4502-a08c-fd0237183f72 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:11.906636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:11.929191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a79ef-af47-4314-b3ee-54f65ef4e0ba None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:11.959203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2382f-a21f-4c6f-9f9a-2288b2ba840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:11.960067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2382f-a21f-4c6f-9f9a-2288b2ba840b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:11.960596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2382f-a21f-4c6f-9f9a-2288b2ba840b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:11.961011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2382f-a21f-4c6f-9f9a-2288b2ba840b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:11.961958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2382f-a21f-4c6f-9f9a-2288b2ba840b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:11.962454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2382f-a21f-4c6f-9f9a-2288b2ba840b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:11.964006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2382f-a21f-4c6f-9f9a-2288b2ba840b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:11.973802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7d91c-e488-4df2-8fd8-43bb01034c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:11.973802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7d91c-e488-4df2-8fd8-43bb01034c39 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:11.973802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7d91c-e488-4df2-8fd8-43bb01034c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:11.973802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b7d91c-e488-4df2-8fd8-43bb01034c39 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:11.974471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7d91c-e488-4df2-8fd8-43bb01034c39 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:11.976133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b7d91c-e488-4df2-8fd8-43bb01034c39 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:11.978951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 83/167] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:11 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:11.979949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7d91c-e488-4df2-8fd8-43bb01034c39 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:11.985595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 85/168] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:11.986688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2382f-a21f-4c6f-9f9a-2288b2ba840b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:12.103983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b7d91c-e488-4df2-8fd8-43bb01034c39 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:12.145865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919dd5d-b298-4b4c-b9db-b23135ab00be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:12.146942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919dd5d-b298-4b4c-b9db-b23135ab00be None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:12.147627 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919dd5d-b298-4b4c-b9db-b23135ab00be None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:12.149757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919dd5d-b298-4b4c-b9db-b23135ab00be None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:12.150874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919dd5d-b298-4b4c-b9db-b23135ab00be None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:12.151291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919dd5d-b298-4b4c-b9db-b23135ab00be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:12.152845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919dd5d-b298-4b4c-b9db-b23135ab00be None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:12.165172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 84/169] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:12.166398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919dd5d-b298-4b4c-b9db-b23135ab00be None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:12.186203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:12.293111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919dd5d-b298-4b4c-b9db-b23135ab00be None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:12.334243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625fc97-ded1-45ec-89d3-0e0c52692e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:12.335385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625fc97-ded1-45ec-89d3-0e0c52692e0e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:12.335385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625fc97-ded1-45ec-89d3-0e0c52692e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:12.335862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625fc97-ded1-45ec-89d3-0e0c52692e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:12.340253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625fc97-ded1-45ec-89d3-0e0c52692e0e None admin] RBAC: Adding query filter params (role.id=f8c927860d6941559283c83fcaaf93d8, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15, user.id=f11433fdb39c41a9b2b8dd15ec05cb21) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:12.340253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625fc97-ded1-45ec-89d3-0e0c52692e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:12.340253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625fc97-ded1-45ec-89d3-0e0c52692e0e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:12.346026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 85/170] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:12 2022] GET /identity/v3/role_assignments?user.id=f11433fdb39c41a9b2b8dd15ec05cb21&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:12.347140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625fc97-ded1-45ec-89d3-0e0c52692e0e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:12.349788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2382f-a21f-4c6f-9f9a-2288b2ba840b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:12.434033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:12.447361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:12.479107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625fc97-ded1-45ec-89d3-0e0c52692e0e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:12.495479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffee0fd-72d3-4d78-beeb-267734ca5f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:12.496118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee0fd-72d3-4d78-beeb-267734ca5f7b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:12.496411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee0fd-72d3-4d78-beeb-267734ca5f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:12.496682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffee0fd-72d3-4d78-beeb-267734ca5f7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:12.523313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9b639-7b31-40f1-b07c-7602fc8e2b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:12.524312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9b639-7b31-40f1-b07c-7602fc8e2b7c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:12.524598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9b639-7b31-40f1-b07c-7602fc8e2b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:12.524875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9b639-7b31-40f1-b07c-7602fc8e2b7c None admin] PATH_INFO: `/v3/users/3476c91ad03941919217e0940471d201` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:12.526145 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9b639-7b31-40f1-b07c-7602fc8e2b7c None admin] RBAC: Authorizing `identity:get_user(user_id=3476c91ad03941919217e0940471d201)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:12.527670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9b639-7b31-40f1-b07c-7602fc8e2b7c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:12.529389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 86/171] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:12 2022] GET /identity/v3/users/3476c91ad03941919217e0940471d201 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:12.530312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9b639-7b31-40f1-b07c-7602fc8e2b7c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:12.668987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9b639-7b31-40f1-b07c-7602fc8e2b7c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:12.719915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38c985-8274-46d0-90ca-114289ca6e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:12.720675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38c985-8274-46d0-90ca-114289ca6e86 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:12.720963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38c985-8274-46d0-90ca-114289ca6e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:12.725953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38c985-8274-46d0-90ca-114289ca6e86 None admin] PATH_INFO: `/v3/users/3c71bc88f3914d2bb5ae4e7c3fa1ee8f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:12.725953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38c985-8274-46d0-90ca-114289ca6e86 None admin] RBAC: Authorizing `identity:get_user(user_id=3c71bc88f3914d2bb5ae4e7c3fa1ee8f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:12.725953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38c985-8274-46d0-90ca-114289ca6e86 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:12.725953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 87/172] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:12 2022] GET /identity/v3/users/3c71bc88f3914d2bb5ae4e7c3fa1ee8f => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:12.726952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38c985-8274-46d0-90ca-114289ca6e86 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:12.731252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffee0fd-72d3-4d78-beeb-267734ca5f7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:12.737802 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffee0fd-72d3-4d78-beeb-267734ca5f7b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:12.744745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:12.915482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38c985-8274-46d0-90ca-114289ca6e86 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:12.955632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cf55b-05ef-45c6-8bdd-7a7c40a6fcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:12.956318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cf55b-05ef-45c6-8bdd-7a7c40a6fcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:12.956538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cf55b-05ef-45c6-8bdd-7a7c40a6fcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:12.956812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cf55b-05ef-45c6-8bdd-7a7c40a6fcc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:13.061373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 86/173] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 1075 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:13.063776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffee0fd-72d3-4d78-beeb-267734ca5f7b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:13.123309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cf55b-05ef-45c6-8bdd-7a7c40a6fcc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:13.126342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cf55b-05ef-45c6-8bdd-7a7c40a6fcc1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:13.157374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:13.174059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:13.434748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffee0fd-72d3-4d78-beeb-267734ca5f7b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:13.451488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 88/174] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:13.453439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037fef1-8166-4e04-bfa9-e57361ea6b09 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:13.453732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037fef1-8166-4e04-bfa9-e57361ea6b09 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:13.458935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037fef1-8166-4e04-bfa9-e57361ea6b09 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:13.467266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 89/175] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:13 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:13.468267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037fef1-8166-4e04-bfa9-e57361ea6b09 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:13.523104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8ac0e-bd0f-4763-88e5-cc9f5ef7da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:13.523768 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8ac0e-bd0f-4763-88e5-cc9f5ef7da68 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:13.524041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8ac0e-bd0f-4763-88e5-cc9f5ef7da68 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:13.524313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8ac0e-bd0f-4763-88e5-cc9f5ef7da68 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:13.525528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8ac0e-bd0f-4763-88e5-cc9f5ef7da68 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:13.527075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8ac0e-bd0f-4763-88e5-cc9f5ef7da68 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:13.529357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 87/176] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:13 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 467 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:13.530011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8ac0e-bd0f-4763-88e5-cc9f5ef7da68 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:13.645542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037fef1-8166-4e04-bfa9-e57361ea6b09 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:13.674645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:13.675100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:13.699040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8ac0e-bd0f-4763-88e5-cc9f5ef7da68 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:13.714522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a14ce5-bf1f-4696-8753-5ccd35f5d99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:13.715560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14ce5-bf1f-4696-8753-5ccd35f5d99f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:13.716015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14ce5-bf1f-4696-8753-5ccd35f5d99f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:13.716411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14ce5-bf1f-4696-8753-5ccd35f5d99f None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:13.718942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a14ce5-bf1f-4696-8753-5ccd35f5d99f None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:13.721697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a14ce5-bf1f-4696-8753-5ccd35f5d99f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:13.731592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 90/177] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:13 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:13.731592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d456a65-4314-426e-a04f-71c55ad05a5f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:13.731592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d456a65-4314-426e-a04f-71c55ad05a5f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:13.731592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d456a65-4314-426e-a04f-71c55ad05a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:13.752238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4d456a65-4314-426e-a04f-71c55ad05a5f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:13.754126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4d456a65-4314-426e-a04f-71c55ad05a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:13.768198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ede1077-94ef-4bf1-a7a2-676c711b624d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:13.768992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede1077-94ef-4bf1-a7a2-676c711b624d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:13.769489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede1077-94ef-4bf1-a7a2-676c711b624d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:13.769979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ede1077-94ef-4bf1-a7a2-676c711b624d None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:13.771846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede1077-94ef-4bf1-a7a2-676c711b624d None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:13.773878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ede1077-94ef-4bf1-a7a2-676c711b624d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:13.776732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 88/178] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:13 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:13.779753 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e543d0b-ae0d-45f5-8961-32aff31c8bb8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:13.780247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e543d0b-ae0d-45f5-8961-32aff31c8bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:13.780728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e543d0b-ae0d-45f5-8961-32aff31c8bb8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:13.783735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 89/179] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:13.785305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e543d0b-ae0d-45f5-8961-32aff31c8bb8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:13.880689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 91/180] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:13.881656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d456a65-4314-426e-a04f-71c55ad05a5f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:13.946004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e543d0b-ae0d-45f5-8961-32aff31c8bb8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:14.006750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32017cc-96cd-4e7b-9916-2fd16eb878ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:14.007907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32017cc-96cd-4e7b-9916-2fd16eb878ec None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:14.008457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32017cc-96cd-4e7b-9916-2fd16eb878ec None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:14.008956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32017cc-96cd-4e7b-9916-2fd16eb878ec None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/users/3c71bc88f3914d2bb5ae4e7c3fa1ee8f/roles/4bbefa34d6764a6fa5b84ff6d1a4f09f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:14.012944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32017cc-96cd-4e7b-9916-2fd16eb878ec None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd6e674b6d240749abdcbbd0b748d15, user_id=3c71bc88f3914d2bb5ae4e7c3fa1ee8f, role_id=4bbefa34d6764a6fa5b84ff6d1a4f09f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:14.015912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32017cc-96cd-4e7b-9916-2fd16eb878ec None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:14.041468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 90/181] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:54:13 2022] PUT /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/users/3c71bc88f3914d2bb5ae4e7c3fa1ee8f/roles/4bbefa34d6764a6fa5b84ff6d1a4f09f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:14.043015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32017cc-96cd-4e7b-9916-2fd16eb878ec None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:14.044230 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d456a65-4314-426e-a04f-71c55ad05a5f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:14.075819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:14.090851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:14.139023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbcc190-740c-43fc-acc7-3121c11c683d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:14.140041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcc190-740c-43fc-acc7-3121c11c683d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:14.141137 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcc190-740c-43fc-acc7-3121c11c683d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:14.141642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbcc190-740c-43fc-acc7-3121c11c683d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:14.143169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbcc190-740c-43fc-acc7-3121c11c683d None admin] RBAC: Adding query filter params (role.id=a631c57af3494b9c955799e0488518ae, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15, user.id=f11433fdb39c41a9b2b8dd15ec05cb21) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:14.143727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbcc190-740c-43fc-acc7-3121c11c683d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:14.146885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbcc190-740c-43fc-acc7-3121c11c683d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:14.162217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 92/182] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:13 2022] GET /identity/v3/role_assignments?user.id=f11433fdb39c41a9b2b8dd15ec05cb21&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=a631c57af3494b9c955799e0488518ae => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:14.164642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf43c0-d5df-471e-9070-865cb7350bf1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:14.165140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf43c0-d5df-471e-9070-865cb7350bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:14.165639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbf43c0-d5df-471e-9070-865cb7350bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:14.203047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4cbf43c0-d5df-471e-9070-865cb7350bf1 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:14.205610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4cbf43c0-d5df-471e-9070-865cb7350bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:14.294326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32017cc-96cd-4e7b-9916-2fd16eb878ec None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:14.323682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 93/183] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:14.326048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdaa24-9093-4870-a276-42df66ceee56 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:14.332269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdaa24-9093-4870-a276-42df66ceee56 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:14.332650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdaa24-9093-4870-a276-42df66ceee56 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:14.334493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 94/184] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:14.335276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdaa24-9093-4870-a276-42df66ceee56 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:14.362420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48ac5f-fadc-4d62-8f9b-46b2f21dcd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:14.363452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48ac5f-fadc-4d62-8f9b-46b2f21dcd5c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:14.363957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48ac5f-fadc-4d62-8f9b-46b2f21dcd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:14.364462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48ac5f-fadc-4d62-8f9b-46b2f21dcd5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:14.365916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48ac5f-fadc-4d62-8f9b-46b2f21dcd5c None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15, user.id=3476c91ad03941919217e0940471d201) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:14.366490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48ac5f-fadc-4d62-8f9b-46b2f21dcd5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:14.369351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48ac5f-fadc-4d62-8f9b-46b2f21dcd5c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:14.385439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 91/185] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:14 2022] GET /identity/v3/role_assignments?user.id=3476c91ad03941919217e0940471d201&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=8b350845f331423cb0494e3449b58310 => generated 608 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:14.385439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb7e03-cefd-4e3e-8dc9-e99a8c420ff5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:14.385439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb7e03-cefd-4e3e-8dc9-e99a8c420ff5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:14.385439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb7e03-cefd-4e3e-8dc9-e99a8c420ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:14.416569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cfdb7e03-cefd-4e3e-8dc9-e99a8c420ff5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:14.418801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cfdb7e03-cefd-4e3e-8dc9-e99a8c420ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:14.581392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdaa24-9093-4870-a276-42df66ceee56 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:14.591526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:14.606238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 92/186] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:14.654145 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0464daf-4d38-4efe-9c59-9135ac6d49c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:14.655239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0464daf-4d38-4efe-9c59-9135ac6d49c2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:14.655746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0464daf-4d38-4efe-9c59-9135ac6d49c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:14.656855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0464daf-4d38-4efe-9c59-9135ac6d49c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:14.660151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0464daf-4d38-4efe-9c59-9135ac6d49c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:14.662722 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0464daf-4d38-4efe-9c59-9135ac6d49c2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:14.666126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 95/187] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:54:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:14.677987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb7e03-cefd-4e3e-8dc9-e99a8c420ff5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:14.695680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0464daf-4d38-4efe-9c59-9135ac6d49c2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:14.704686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:14.861491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0464daf-4d38-4efe-9c59-9135ac6d49c2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:14.887790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb7e03-cefd-4e3e-8dc9-e99a8c420ff5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:14.937119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423ef97-59b5-4b29-b887-5f28fee1742e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:14.938566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423ef97-59b5-4b29-b887-5f28fee1742e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:14.939344 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423ef97-59b5-4b29-b887-5f28fee1742e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:14.940091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423ef97-59b5-4b29-b887-5f28fee1742e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:14.964714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423ef97-59b5-4b29-b887-5f28fee1742e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a423ef97-59b5-4b29-b887-5f28fee1742e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:14.976125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:14.977745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:14.979422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:14.980600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5fa6a-9036-4638-87bb-56e8e1c5fdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:14.980600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5fa6a-9036-4638-87bb-56e8e1c5fdf6 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:14.980600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5fa6a-9036-4638-87bb-56e8e1c5fdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:14.980600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5fa6a-9036-4638-87bb-56e8e1c5fdf6 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:14.982119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5fa6a-9036-4638-87bb-56e8e1c5fdf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:14.982119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5fa6a-9036-4638-87bb-56e8e1c5fdf6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:14.985299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 96/188] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:54:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:14.995394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423ef97-59b5-4b29-b887-5f28fee1742e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:15.016649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 93/189] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:54:14 2022] POST /identity/v3/users => generated 297 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:54:15.023433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5fa6a-9036-4638-87bb-56e8e1c5fdf6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:15.057827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:15.163809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:15.201823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423ef97-59b5-4b29-b887-5f28fee1742e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:15.226061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5fa6a-9036-4638-87bb-56e8e1c5fdf6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:15.266580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe952c1-54b9-43e4-9fa4-e3dba1822b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:15.267280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe952c1-54b9-43e4-9fa4-e3dba1822b96 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:15.267563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe952c1-54b9-43e4-9fa4-e3dba1822b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:15.267834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe952c1-54b9-43e4-9fa4-e3dba1822b96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:15.270843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0a138-5c63-403d-a80c-376df0626f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:15.272299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0a138-5c63-403d-a80c-376df0626f84 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:15.273095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0a138-5c63-403d-a80c-376df0626f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:15.273844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0a138-5c63-403d-a80c-376df0626f84 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:15.274755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0a138-5c63-403d-a80c-376df0626f84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:15.275097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0a138-5c63-403d-a80c-376df0626f84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:15.276535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0a138-5c63-403d-a80c-376df0626f84 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:15.286035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe952c1-54b9-43e4-9fa4-e3dba1822b96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:15.291678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 97/190] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-abe952c1-54b9-43e4-9fa4-e3dba1822b96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:15.292612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:15.293800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:15.294609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 94/191] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:15.300832 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0a138-5c63-403d-a80c-376df0626f84 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:15.302765 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe952c1-54b9-43e4-9fa4-e3dba1822b96 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:15.455982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe952c1-54b9-43e4-9fa4-e3dba1822b96 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:15.466310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:15.476574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0a138-5c63-403d-a80c-376df0626f84 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:15.509803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:15.517256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d8006-eb93-40d3-9f9d-9cb84ff4821d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:15.519258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d8006-eb93-40d3-9f9d-9cb84ff4821d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:15.520340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d8006-eb93-40d3-9f9d-9cb84ff4821d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:15.520717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d8006-eb93-40d3-9f9d-9cb84ff4821d None admin] PATH_INFO: `/v3/users/541c746bdc1a458b9036557defa4e094` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:15.524445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d454f3-7396-41d2-b4b1-7b1d6caa6e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:15.525082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d454f3-7396-41d2-b4b1-7b1d6caa6e2b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:15.525357 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d454f3-7396-41d2-b4b1-7b1d6caa6e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:15.525627 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d454f3-7396-41d2-b4b1-7b1d6caa6e2b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:15.526483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d454f3-7396-41d2-b4b1-7b1d6caa6e2b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:15.526794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d454f3-7396-41d2-b4b1-7b1d6caa6e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:15.528321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d454f3-7396-41d2-b4b1-7b1d6caa6e2b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:15.538846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d8006-eb93-40d3-9f9d-9cb84ff4821d None admin] RBAC: Authorizing `identity:get_user(user_id=541c746bdc1a458b9036557defa4e094)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:15.539300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 98/192] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:15.540373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d8006-eb93-40d3-9f9d-9cb84ff4821d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:15.542420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 95/193] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:15 2022] GET /identity/v3/users/541c746bdc1a458b9036557defa4e094 => generated 283 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:15.544913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d454f3-7396-41d2-b4b1-7b1d6caa6e2b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:15.548081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d8006-eb93-40d3-9f9d-9cb84ff4821d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:15.704098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d454f3-7396-41d2-b4b1-7b1d6caa6e2b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:15.719963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d8006-eb93-40d3-9f9d-9cb84ff4821d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:15.762277 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b2d28-3800-47e2-8d35-350facf45c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:15.762929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b2d28-3800-47e2-8d35-350facf45c13 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:15.763207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b2d28-3800-47e2-8d35-350facf45c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:15.763480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638b2d28-3800-47e2-8d35-350facf45c13 None admin] PATH_INFO: `/v3/users/541c746bdc1a458b9036557defa4e094` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:15.768076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b2d28-3800-47e2-8d35-350facf45c13 None admin] RBAC: Authorizing `identity:get_user(user_id=541c746bdc1a458b9036557defa4e094)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:15.769560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638b2d28-3800-47e2-8d35-350facf45c13 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:15.771471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 99/194] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:15 2022] GET /identity/v3/users/541c746bdc1a458b9036557defa4e094 => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:15.777614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc6c2-ef08-43ae-9bf8-aed9e4e63c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:15.778273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc6c2-ef08-43ae-9bf8-aed9e4e63c4c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:15.778548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc6c2-ef08-43ae-9bf8-aed9e4e63c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:15.778818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebc6c2-ef08-43ae-9bf8-aed9e4e63c4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:15.780352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b2d28-3800-47e2-8d35-350facf45c13 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:15.860049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:15.928454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:15.972024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebc6c2-ef08-43ae-9bf8-aed9e4e63c4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:15.974964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebc6c2-ef08-43ae-9bf8-aed9e4e63c4c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:16.010126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638b2d28-3800-47e2-8d35-350facf45c13 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:16.056182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf17862-b469-412d-bf9b-25e977c1e28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:16.056971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17862-b469-412d-bf9b-25e977c1e28e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:16.057358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17862-b469-412d-bf9b-25e977c1e28e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:16.057739 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf17862-b469-412d-bf9b-25e977c1e28e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:16.142294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf17862-b469-412d-bf9b-25e977c1e28e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:16.143837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf17862-b469-412d-bf9b-25e977c1e28e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:16.171986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:16.266885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 96/195] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 20:54:16.274189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc6c2-ef08-43ae-9bf8-aed9e4e63c4c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:16.277894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 100/196] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 20:54:16.283577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf17862-b469-412d-bf9b-25e977c1e28e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:16.436876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf17862-b469-412d-bf9b-25e977c1e28e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:16.469078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:16.482091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a83abb-6da8-4f34-b99b-efc54dcc7107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:16.482750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a83abb-6da8-4f34-b99b-efc54dcc7107 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:16.483031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a83abb-6da8-4f34-b99b-efc54dcc7107 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:16.483309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a83abb-6da8-4f34-b99b-efc54dcc7107 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:16.484716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a83abb-6da8-4f34-b99b-efc54dcc7107 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:16.486385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a83abb-6da8-4f34-b99b-efc54dcc7107 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:16.488465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 101/197] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:16 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:16.497297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a83abb-6da8-4f34-b99b-efc54dcc7107 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:16.514219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:16.577694 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebc6c2-ef08-43ae-9bf8-aed9e4e63c4c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:16.636977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ec320-d7ba-4d16-afa4-da62f04204cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:16.637631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ec320-d7ba-4d16-afa4-da62f04204cd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:16.637937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ec320-d7ba-4d16-afa4-da62f04204cd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:16.638212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ec320-d7ba-4d16-afa4-da62f04204cd None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:16.639389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ec320-d7ba-4d16-afa4-da62f04204cd None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:16.641402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ec320-d7ba-4d16-afa4-da62f04204cd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:16.643667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 97/198] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:16 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:16.649412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ec320-d7ba-4d16-afa4-da62f04204cd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:16.762318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ec320-d7ba-4d16-afa4-da62f04204cd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:16.815493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100d498-9c0b-4811-bc29-9bd80267cf72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:16.816353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100d498-9c0b-4811-bc29-9bd80267cf72 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:16.816743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100d498-9c0b-4811-bc29-9bd80267cf72 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:16.817122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100d498-9c0b-4811-bc29-9bd80267cf72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:16.818245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100d498-9c0b-4811-bc29-9bd80267cf72 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=4ff250f07e37455ba8616d6434ffbef6, user.id=541c746bdc1a458b9036557defa4e094) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:16.818700 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100d498-9c0b-4811-bc29-9bd80267cf72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:16.821026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100d498-9c0b-4811-bc29-9bd80267cf72 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:16.826147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a83abb-6da8-4f34-b99b-efc54dcc7107 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:16.848734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 98/199] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:16 2022] GET /identity/v3/role_assignments?user.id=541c746bdc1a458b9036557defa4e094&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=8b350845f331423cb0494e3449b58310 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:16.906113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b3aeb-4da8-41fa-b19b-81a2affa091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:16.907179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b3aeb-4da8-41fa-b19b-81a2affa091d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:16.907660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b3aeb-4da8-41fa-b19b-81a2affa091d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:16.908133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b3aeb-4da8-41fa-b19b-81a2affa091d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:16.909559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b3aeb-4da8-41fa-b19b-81a2affa091d None admin] RBAC: Adding query filter params (role.id=a631c57af3494b9c955799e0488518ae, scope.project.id=4ff250f07e37455ba8616d6434ffbef6, user.id=541c746bdc1a458b9036557defa4e094) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:16.910158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b3aeb-4da8-41fa-b19b-81a2affa091d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:16.913026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b3aeb-4da8-41fa-b19b-81a2affa091d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:16.925434 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 102/200] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:16 2022] GET /identity/v3/role_assignments?user.id=541c746bdc1a458b9036557defa4e094&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=a631c57af3494b9c955799e0488518ae => generated 259 bytes in 429 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:16.930233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ce79d-ee5b-489e-984c-75608ac4d6de None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:16.930518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ce79d-ee5b-489e-984c-75608ac4d6de None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:16.930786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ce79d-ee5b-489e-984c-75608ac4d6de None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:16.932170 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 99/201] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:54:16.939248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f2e9c-d663-4e92-9fcb-ab6c7220c37a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:16.939571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f2e9c-d663-4e92-9fcb-ab6c7220c37a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:16.939802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f2e9c-d663-4e92-9fcb-ab6c7220c37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:16.960532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ea1f2e9c-d663-4e92-9fcb-ab6c7220c37a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:16.961821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ea1f2e9c-d663-4e92-9fcb-ab6c7220c37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:17.033815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:17.044955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 103/202] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:17.322343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ce79d-ee5b-489e-984c-75608ac4d6de None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:17.346048 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:17.491425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ce79d-ee5b-489e-984c-75608ac4d6de None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:17.543264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63334801-c2bd-47b5-b105-9bc3bd5c4e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:17.543912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63334801-c2bd-47b5-b105-9bc3bd5c4e87 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:17.544185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63334801-c2bd-47b5-b105-9bc3bd5c4e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:17.544465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63334801-c2bd-47b5-b105-9bc3bd5c4e87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:17.569442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63334801-c2bd-47b5-b105-9bc3bd5c4e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-63334801-c2bd-47b5-b105-9bc3bd5c4e87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:17.580510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:17.582272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:17.583856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:17.586779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 100/203] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:17.592736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f2e9c-d663-4e92-9fcb-ab6c7220c37a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:17.774511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f2e9c-d663-4e92-9fcb-ab6c7220c37a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:17.863231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f393f3a-fa10-42e1-9e9a-4744fc755c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:17.865608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f393f3a-fa10-42e1-9e9a-4744fc755c6e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:17.866406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f393f3a-fa10-42e1-9e9a-4744fc755c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:17.867198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f393f3a-fa10-42e1-9e9a-4744fc755c6e None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:17.869006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f393f3a-fa10-42e1-9e9a-4744fc755c6e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:17.869881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f393f3a-fa10-42e1-9e9a-4744fc755c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:17.872866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f393f3a-fa10-42e1-9e9a-4744fc755c6e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:17.897348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 104/204] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:17.903101 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63334801-c2bd-47b5-b105-9bc3bd5c4e87 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:17.981533 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038729b-e597-4313-9944-edeaa5770eae None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:17.983301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038729b-e597-4313-9944-edeaa5770eae None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:17.984203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038729b-e597-4313-9944-edeaa5770eae None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:17.989324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 105/205] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:17.996305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df9dd-6416-4741-a712-024b0083feca None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:17.997200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df9dd-6416-4741-a712-024b0083feca None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:17.998072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14df9dd-6416-4741-a712-024b0083feca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:18.041828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d14df9dd-6416-4741-a712-024b0083feca None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:18.045068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d14df9dd-6416-4741-a712-024b0083feca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:18.102617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:18.156900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63334801-c2bd-47b5-b105-9bc3bd5c4e87 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:18.178202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:18.228764 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830430a-b19a-4000-b141-96b600ab14f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:18.230022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830430a-b19a-4000-b141-96b600ab14f4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:18.230516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830430a-b19a-4000-b141-96b600ab14f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:18.231134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830430a-b19a-4000-b141-96b600ab14f4 None admin] PATH_INFO: `/v3/users/3c71bc88f3914d2bb5ae4e7c3fa1ee8f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:18.233268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830430a-b19a-4000-b141-96b600ab14f4 None admin] RBAC: Authorizing `identity:get_user(user_id=3c71bc88f3914d2bb5ae4e7c3fa1ee8f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:18.236995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2830430a-b19a-4000-b141-96b600ab14f4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:18.237144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 106/206] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:18.240340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 101/207] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:17 2022] GET /identity/v3/users/3c71bc88f3914d2bb5ae4e7c3fa1ee8f => generated 289 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:18.250049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14df9dd-6416-4741-a712-024b0083feca None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:18.377663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14df9dd-6416-4741-a712-024b0083feca None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:18.430012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50b0e8-018b-4dfe-8975-7bb494aa0ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:18.431138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50b0e8-018b-4dfe-8975-7bb494aa0ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:18.431624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50b0e8-018b-4dfe-8975-7bb494aa0ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:18.432109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50b0e8-018b-4dfe-8975-7bb494aa0ac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:18.489964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8261bd-e406-4aaf-b39b-d44008abe887 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:18.490413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8261bd-e406-4aaf-b39b-d44008abe887 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:18.490859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8261bd-e406-4aaf-b39b-d44008abe887 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:18.493891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 102/208] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:18.500726 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86986862-87ff-4081-890f-09d61304340c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:18.501272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86986862-87ff-4081-890f-09d61304340c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:18.501749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86986862-87ff-4081-890f-09d61304340c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:18.541672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:18.555688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-86986862-87ff-4081-890f-09d61304340c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:18.557558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-86986862-87ff-4081-890f-09d61304340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:18.599653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50b0e8-018b-4dfe-8975-7bb494aa0ac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:18.603768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be50b0e8-018b-4dfe-8975-7bb494aa0ac1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:18.679331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 103/209] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:18.680324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86986862-87ff-4081-890f-09d61304340c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:18.834203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:18.888661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86986862-87ff-4081-890f-09d61304340c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:18.916810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 107/210] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 667 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 20:54:18.924827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50b0e8-018b-4dfe-8975-7bb494aa0ac1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:18.985528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f16eaf-6908-446c-8190-4f00a09061a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:18.989874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f16eaf-6908-446c-8190-4f00a09061a6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:18.992957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f16eaf-6908-446c-8190-4f00a09061a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:18.993992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f16eaf-6908-446c-8190-4f00a09061a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:18.998091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f16eaf-6908-446c-8190-4f00a09061a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:19.003728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f16eaf-6908-446c-8190-4f00a09061a6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:19.007248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 104/211] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:54:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:19.015989 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f16eaf-6908-446c-8190-4f00a09061a6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:19.066642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:19.096969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be50b0e8-018b-4dfe-8975-7bb494aa0ac1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:19.136973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fdbf0-7a17-415d-be46-09385a6f2f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:19.138090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fdbf0-7a17-415d-be46-09385a6f2f75 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:19.138597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fdbf0-7a17-415d-be46-09385a6f2f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:19.139067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550fdbf0-7a17-415d-be46-09385a6f2f75 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:19.140600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fdbf0-7a17-415d-be46-09385a6f2f75 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:19.142313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550fdbf0-7a17-415d-be46-09385a6f2f75 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:19.144237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 108/212] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:18 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:19.153418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fdbf0-7a17-415d-be46-09385a6f2f75 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:19.272424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f16eaf-6908-446c-8190-4f00a09061a6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:19.279850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550fdbf0-7a17-415d-be46-09385a6f2f75 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:19.323352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe444b4-850b-42a3-bd2f-c79efa852968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:19.324388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe444b4-850b-42a3-bd2f-c79efa852968 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:19.324894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe444b4-850b-42a3-bd2f-c79efa852968 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:19.325366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe444b4-850b-42a3-bd2f-c79efa852968 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:19.339263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe444b4-850b-42a3-bd2f-c79efa852968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:19.339981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed45614-d63c-4b4e-96b8-3e03a28fdf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:19.341023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45614-d63c-4b4e-96b8-3e03a28fdf1d None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:19.341513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45614-d63c-4b4e-96b8-3e03a28fdf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:19.342044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed45614-d63c-4b4e-96b8-3e03a28fdf1d None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:19.343294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed45614-d63c-4b4e-96b8-3e03a28fdf1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1fe444b4-850b-42a3-bd2f-c79efa852968 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:19.345954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:19.346837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:19.347600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:19.348194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed45614-d63c-4b4e-96b8-3e03a28fdf1d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:19.355132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 109/213] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:19.356449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe444b4-850b-42a3-bd2f-c79efa852968 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:19.368433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:19.392794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 105/214] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:54:19 2022] POST /identity/v3/users => generated 297 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:54:19.394840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed45614-d63c-4b4e-96b8-3e03a28fdf1d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:19.418363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:19.622716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed45614-d63c-4b4e-96b8-3e03a28fdf1d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:19.658173 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe444b4-850b-42a3-bd2f-c79efa852968 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:19.701194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b8ed41-a2f3-4dd1-b92e-d85fcf7dc3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:19.702963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8ed41-a2f3-4dd1-b92e-d85fcf7dc3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:19.704211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8ed41-a2f3-4dd1-b92e-d85fcf7dc3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:19.705056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b8ed41-a2f3-4dd1-b92e-d85fcf7dc3f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:19.706652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8ed41-a2f3-4dd1-b92e-d85fcf7dc3f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:19.707592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8ed41-a2f3-4dd1-b92e-d85fcf7dc3f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:19.711003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b8ed41-a2f3-4dd1-b92e-d85fcf7dc3f2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:19.718873 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265ccba-86e8-4a29-98c3-04e8a2c555f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:19.719663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265ccba-86e8-4a29-98c3-04e8a2c555f6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:19.720143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265ccba-86e8-4a29-98c3-04e8a2c555f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:19.720523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265ccba-86e8-4a29-98c3-04e8a2c555f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:19.721387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265ccba-86e8-4a29-98c3-04e8a2c555f6 None admin] RBAC: Adding query filter params (role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15, user.id=3c71bc88f3914d2bb5ae4e7c3fa1ee8f) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:19.721844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265ccba-86e8-4a29-98c3-04e8a2c555f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:19.723972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265ccba-86e8-4a29-98c3-04e8a2c555f6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:19.730642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 110/215] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:19 2022] GET /identity/v3/role_assignments?user.id=3c71bc88f3914d2bb5ae4e7c3fa1ee8f&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f => generated 608 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:19.731510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 106/216] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:19.741100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265ccba-86e8-4a29-98c3-04e8a2c555f6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:19.856674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a05f8-e210-4885-bc49-9558bd4573bb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:19.857150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a05f8-e210-4885-bc49-9558bd4573bb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:19.857633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a05f8-e210-4885-bc49-9558bd4573bb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:19.860493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 107/217] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:19.867764 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4504763-0429-43b4-a683-f4a1480ab14c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:19.868485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4504763-0429-43b4-a683-f4a1480ab14c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:19.868981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4504763-0429-43b4-a683-f4a1480ab14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:19.886462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:19.907906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f4504763-0429-43b4-a683-f4a1480ab14c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:19.909550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f4504763-0429-43b4-a683-f4a1480ab14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:19.972203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265ccba-86e8-4a29-98c3-04e8a2c555f6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:19.992196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:20.010190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 108/218] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:20.055045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c8852-c275-4ac6-a4bd-0f6ef84b88f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:20.056173 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8852-c275-4ac6-a4bd-0f6ef84b88f0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:20.056676 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8852-c275-4ac6-a4bd-0f6ef84b88f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:20.057142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8852-c275-4ac6-a4bd-0f6ef84b88f0 None admin] PATH_INFO: `/v3/users/4a01f1bb14ad4a0dbc5250126de78ece` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:20.091699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c8852-c275-4ac6-a4bd-0f6ef84b88f0 None admin] RBAC: Authorizing `identity:get_user(user_id=4a01f1bb14ad4a0dbc5250126de78ece)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:20.095077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c8852-c275-4ac6-a4bd-0f6ef84b88f0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:20.098286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 111/219] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:19 2022] GET /identity/v3/users/4a01f1bb14ad4a0dbc5250126de78ece => generated 297 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:20.104456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4504763-0429-43b4-a683-f4a1480ab14c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:20.216302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0655860-83c3-4cb8-a6ec-5012e9f92a18 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:20.216541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0655860-83c3-4cb8-a6ec-5012e9f92a18 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:20.216787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0655860-83c3-4cb8-a6ec-5012e9f92a18 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:20.218554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 112/220] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:54:20.226503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f0dc8-c96d-4784-8416-6b02bc0aea36 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:20.227791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f0dc8-c96d-4784-8416-6b02bc0aea36 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:20.227791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f0dc8-c96d-4784-8416-6b02bc0aea36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:20.248199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-690f0dc8-c96d-4784-8416-6b02bc0aea36 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:20.251496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-690f0dc8-c96d-4784-8416-6b02bc0aea36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:20.251658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4504763-0429-43b4-a683-f4a1480ab14c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:20.303834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28451c-900a-4938-a853-aeaa3dbbc8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:20.305085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28451c-900a-4938-a853-aeaa3dbbc8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:20.305576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28451c-900a-4938-a853-aeaa3dbbc8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:20.306223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28451c-900a-4938-a853-aeaa3dbbc8c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:20.333923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 113/221] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:20.378953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:20.391497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28451c-900a-4938-a853-aeaa3dbbc8c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:20.393309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f28451c-900a-4938-a853-aeaa3dbbc8c4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:20.522726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f0dc8-c96d-4784-8416-6b02bc0aea36 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:20.585618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 109/222] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:20.596975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28451c-900a-4938-a853-aeaa3dbbc8c4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:20.646173 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:20.711960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f0dc8-c96d-4784-8416-6b02bc0aea36 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:20.754653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e153c82-dc09-4bca-94cf-a9425c88fc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:20.756872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e153c82-dc09-4bca-94cf-a9425c88fc63 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:20.757258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e153c82-dc09-4bca-94cf-a9425c88fc63 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:20.757608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e153c82-dc09-4bca-94cf-a9425c88fc63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:20.759783 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:20.775043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28451c-900a-4938-a853-aeaa3dbbc8c4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:20.775892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e153c82-dc09-4bca-94cf-a9425c88fc63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1e153c82-dc09-4bca-94cf-a9425c88fc63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:20.782523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:20.783679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:20.784665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:20.790763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 114/223] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:54:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:54:20.790763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e153c82-dc09-4bca-94cf-a9425c88fc63 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:20.831513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39453bb-e97d-48b6-ae75-1112d409cb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:20.832668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39453bb-e97d-48b6-ae75-1112d409cb2a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:20.833554 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39453bb-e97d-48b6-ae75-1112d409cb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:20.834290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39453bb-e97d-48b6-ae75-1112d409cb2a None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:20.835669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39453bb-e97d-48b6-ae75-1112d409cb2a None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:20.838417 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39453bb-e97d-48b6-ae75-1112d409cb2a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:20.844277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 110/224] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:20 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:20.845654 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39453bb-e97d-48b6-ae75-1112d409cb2a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:20.925396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e153c82-dc09-4bca-94cf-a9425c88fc63 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:20.954791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:20.979568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc528f6-c67b-4813-a63e-e516b07ea9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:20.980200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc528f6-c67b-4813-a63e-e516b07ea9cc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:20.980463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc528f6-c67b-4813-a63e-e516b07ea9cc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:20.980733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc528f6-c67b-4813-a63e-e516b07ea9cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:20.995435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39453bb-e97d-48b6-ae75-1112d409cb2a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:20.995941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc528f6-c67b-4813-a63e-e516b07ea9cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bbc528f6-c67b-4813-a63e-e516b07ea9cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:21.002096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:21.003021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:21.003799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 115/225] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:21.004465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc528f6-c67b-4813-a63e-e516b07ea9cc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:21.044530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f4319-b9e2-4983-8484-2b833c5b9f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:21.045191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f4319-b9e2-4983-8484-2b833c5b9f4e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:21.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f4319-b9e2-4983-8484-2b833c5b9f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:21.045624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f4319-b9e2-4983-8484-2b833c5b9f4e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:21.046893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f4319-b9e2-4983-8484-2b833c5b9f4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:21.047291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f4319-b9e2-4983-8484-2b833c5b9f4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:21.049049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f4319-b9e2-4983-8484-2b833c5b9f4e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:21.062873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 111/226] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:21.064326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f4319-b9e2-4983-8484-2b833c5b9f4e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:21.089414 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:21.140753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc528f6-c67b-4813-a63e-e516b07ea9cc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:21.177262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f4319-b9e2-4983-8484-2b833c5b9f4e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:21.179645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d846b1-f46f-4917-9b1a-50405ca46c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:21.180538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d846b1-f46f-4917-9b1a-50405ca46c2b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:21.181021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d846b1-f46f-4917-9b1a-50405ca46c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:21.181496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d846b1-f46f-4917-9b1a-50405ca46c2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:21.182522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d846b1-f46f-4917-9b1a-50405ca46c2b None admin] RBAC: Adding query filter params (role.id=f8c927860d6941559283c83fcaaf93d8, scope.project.id=4ff250f07e37455ba8616d6434ffbef6, user.id=4a01f1bb14ad4a0dbc5250126de78ece) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:21.183082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d846b1-f46f-4917-9b1a-50405ca46c2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:21.184770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d846b1-f46f-4917-9b1a-50405ca46c2b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:21.191658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 116/227] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:21 2022] GET /identity/v3/role_assignments?user.id=4a01f1bb14ad4a0dbc5250126de78ece&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:21.192861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d846b1-f46f-4917-9b1a-50405ca46c2b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:21.219054 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e5b77-42d2-4a88-ac9d-d2541c8c7efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:21.220010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e5b77-42d2-4a88-ac9d-d2541c8c7efd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:21.220283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e5b77-42d2-4a88-ac9d-d2541c8c7efd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:21.220558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e5b77-42d2-4a88-ac9d-d2541c8c7efd None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:21.221244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e5b77-42d2-4a88-ac9d-d2541c8c7efd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:21.221546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e5b77-42d2-4a88-ac9d-d2541c8c7efd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:21.222994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e5b77-42d2-4a88-ac9d-d2541c8c7efd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:21.232752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 112/228] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:21.239103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e5b77-42d2-4a88-ac9d-d2541c8c7efd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:21.251888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:21.336524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d846b1-f46f-4917-9b1a-50405ca46c2b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:21.341056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:21.366477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e5b77-42d2-4a88-ac9d-d2541c8c7efd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:21.380384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b987d7-4caa-4e34-b8f8-878e7155b11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:21.381506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b987d7-4caa-4e34-b8f8-878e7155b11f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:21.382076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b987d7-4caa-4e34-b8f8-878e7155b11f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:21.382601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b987d7-4caa-4e34-b8f8-878e7155b11f None admin] PATH_INFO: `/v3/users/541c746bdc1a458b9036557defa4e094` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:21.384199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b987d7-4caa-4e34-b8f8-878e7155b11f None admin] RBAC: Authorizing `identity:get_user(user_id=541c746bdc1a458b9036557defa4e094)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:21.386051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b987d7-4caa-4e34-b8f8-878e7155b11f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:21.388459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 117/229] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:21 2022] GET /identity/v3/users/541c746bdc1a458b9036557defa4e094 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:21.401380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b987d7-4caa-4e34-b8f8-878e7155b11f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:21.414970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93931d6-a481-4644-a9fd-ff30046c959e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:21.415633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93931d6-a481-4644-a9fd-ff30046c959e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:21.415912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93931d6-a481-4644-a9fd-ff30046c959e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:21.416186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93931d6-a481-4644-a9fd-ff30046c959e None admin] PATH_INFO: `/v3/users/541c746bdc1a458b9036557defa4e094` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:21.417505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93931d6-a481-4644-a9fd-ff30046c959e None admin] RBAC: Authorizing `identity:get_user(user_id=541c746bdc1a458b9036557defa4e094)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:21.419613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93931d6-a481-4644-a9fd-ff30046c959e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:21.420763 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 113/230] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:21 2022] GET /identity/v3/users/541c746bdc1a458b9036557defa4e094 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:21.429948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93931d6-a481-4644-a9fd-ff30046c959e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:21.649393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93931d6-a481-4644-a9fd-ff30046c959e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:21.656305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b987d7-4caa-4e34-b8f8-878e7155b11f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:21.668504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:21.696581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4461a3b-ce61-4a08-8174-09edd3093c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:21.697224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4461a3b-ce61-4a08-8174-09edd3093c9f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:21.697500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4461a3b-ce61-4a08-8174-09edd3093c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:21.697786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4461a3b-ce61-4a08-8174-09edd3093c9f None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:21.699004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4461a3b-ce61-4a08-8174-09edd3093c9f None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:21.700465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4461a3b-ce61-4a08-8174-09edd3093c9f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:21.702232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 118/231] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:21 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:21.709127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4461a3b-ce61-4a08-8174-09edd3093c9f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:21.742939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea08a3-a66c-499a-9d96-3a0298babcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:21.744324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea08a3-a66c-499a-9d96-3a0298babcbf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:21.744961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea08a3-a66c-499a-9d96-3a0298babcbf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:21.745457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea08a3-a66c-499a-9d96-3a0298babcbf None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:21.755408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea08a3-a66c-499a-9d96-3a0298babcbf None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:21.758581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea08a3-a66c-499a-9d96-3a0298babcbf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:21.762091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 114/232] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:21 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:21.771304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea08a3-a66c-499a-9d96-3a0298babcbf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:21.835129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4461a3b-ce61-4a08-8174-09edd3093c9f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:21.837621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:21.943478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea08a3-a66c-499a-9d96-3a0298babcbf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:21.967266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb135a-3cb5-4564-af43-b80ee9b73a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:21.967998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb135a-3cb5-4564-af43-b80ee9b73a3a None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:21.968820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb135a-3cb5-4564-af43-b80ee9b73a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:21.969100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb135a-3cb5-4564-af43-b80ee9b73a3a None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6/users/541c746bdc1a458b9036557defa4e094/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:21.971410 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb135a-3cb5-4564-af43-b80ee9b73a3a None admin] RBAC: Authorizing `identity:create_grant(project_id=4ff250f07e37455ba8616d6434ffbef6, user_id=541c746bdc1a458b9036557defa4e094, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:21.972906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb135a-3cb5-4564-af43-b80ee9b73a3a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:21.992850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71831d1-7783-4a9a-831b-18eb0cf4e478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:21.993890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71831d1-7783-4a9a-831b-18eb0cf4e478 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:21.994266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71831d1-7783-4a9a-831b-18eb0cf4e478 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:21.997415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71831d1-7783-4a9a-831b-18eb0cf4e478 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6/users/541c746bdc1a458b9036557defa4e094/roles/a631c57af3494b9c955799e0488518ae` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:22.003070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 119/233] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:54:21 2022] PUT /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6/users/541c746bdc1a458b9036557defa4e094/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 20:54:22.005502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71831d1-7783-4a9a-831b-18eb0cf4e478 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ff250f07e37455ba8616d6434ffbef6, user_id=541c746bdc1a458b9036557defa4e094, role_id=a631c57af3494b9c955799e0488518ae)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:22.007202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71831d1-7783-4a9a-831b-18eb0cf4e478 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:22.042213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 115/234] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:54:21 2022] PUT /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6/users/541c746bdc1a458b9036557defa4e094/roles/a631c57af3494b9c955799e0488518ae => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:22.522707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9542f95-10c8-4d87-a5fe-5e0c010ddf29 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:22.523403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9542f95-10c8-4d87-a5fe-5e0c010ddf29 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:22.523888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9542f95-10c8-4d87-a5fe-5e0c010ddf29 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:22.525481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 120/235] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:22.531931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a70b1c-8a0c-4c4f-90ed-211888bb1b18 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:22.532562 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a70b1c-8a0c-4c4f-90ed-211888bb1b18 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:22.533037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a70b1c-8a0c-4c4f-90ed-211888bb1b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:22.563417 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-54a70b1c-8a0c-4c4f-90ed-211888bb1b18 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:22.565016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-54a70b1c-8a0c-4c4f-90ed-211888bb1b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:22.653429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2cd7-b60b-487e-bf17-c429b51e3a69 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:22.654135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2cd7-b60b-487e-bf17-c429b51e3a69 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:22.654663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2c2cd7-b60b-487e-bf17-c429b51e3a69 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:22.656332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 121/236] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:22.661794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a228a3-3256-4778-9e6c-fbae3fafd0a3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:22.662387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a228a3-3256-4778-9e6c-fbae3fafd0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:22.662902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a228a3-3256-4778-9e6c-fbae3fafd0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:22.696690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 116/237] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:22.698109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-41a228a3-3256-4778-9e6c-fbae3fafd0a3 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:22.700345 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-41a228a3-3256-4778-9e6c-fbae3fafd0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:22.775693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:22.859132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 122/238] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:23.061298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a70b1c-8a0c-4c4f-90ed-211888bb1b18 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:23.082548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:23.229105 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a70b1c-8a0c-4c4f-90ed-211888bb1b18 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:23.279562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a228a3-3256-4778-9e6c-fbae3fafd0a3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:23.280077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cbfd3-f7eb-449b-93ce-13c4894fa849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:23.280805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cbfd3-f7eb-449b-93ce-13c4894fa849 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:23.281837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cbfd3-f7eb-449b-93ce-13c4894fa849 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:23.282128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9cbfd3-f7eb-449b-93ce-13c4894fa849 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:23.285794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cbfd3-f7eb-449b-93ce-13c4894fa849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:23.285794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9cbfd3-f7eb-449b-93ce-13c4894fa849 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:23.287014 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 117/239] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:54:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:23.292696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cbfd3-f7eb-449b-93ce-13c4894fa849 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:23.379077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:23.404929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9cbfd3-f7eb-449b-93ce-13c4894fa849 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:23.465820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a228a3-3256-4778-9e6c-fbae3fafd0a3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:23.483881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b888d2-3f34-4558-ac75-cce1e8176cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:23.484964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b888d2-3f34-4558-ac75-cce1e8176cd4 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:23.485444 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b888d2-3f34-4558-ac75-cce1e8176cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:23.485942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b888d2-3f34-4558-ac75-cce1e8176cd4 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:23.486826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b888d2-3f34-4558-ac75-cce1e8176cd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:23.488537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b888d2-3f34-4558-ac75-cce1e8176cd4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:23.537465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 118/240] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:54:23 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 20:54:23.546733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b375a0-82bb-4051-ad5c-78db2612265a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:23.547568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b375a0-82bb-4051-ad5c-78db2612265a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:23.547849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b375a0-82bb-4051-ad5c-78db2612265a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:23.548225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b375a0-82bb-4051-ad5c-78db2612265a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:23.562765 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b375a0-82bb-4051-ad5c-78db2612265a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f2b375a0-82bb-4051-ad5c-78db2612265a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:23.568048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:23.568926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:23.569698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 123/241] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:23.579379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b888d2-3f34-4558-ac75-cce1e8176cd4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:23.722283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b888d2-3f34-4558-ac75-cce1e8176cd4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:23.759164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:23.769013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643da19b-8b30-492e-86e3-2ff941fac2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:23.769657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643da19b-8b30-492e-86e3-2ff941fac2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:23.769984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643da19b-8b30-492e-86e3-2ff941fac2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:23.770324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643da19b-8b30-492e-86e3-2ff941fac2d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:23.771037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643da19b-8b30-492e-86e3-2ff941fac2d9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:23.771400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643da19b-8b30-492e-86e3-2ff941fac2d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:23.772876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643da19b-8b30-492e-86e3-2ff941fac2d9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:23.784568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 119/242] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:23.792485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b375a0-82bb-4051-ad5c-78db2612265a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:23.830729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:23.907668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b51e45-7867-4e54-bf3e-850dee9d9cda None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:23.907992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b51e45-7867-4e54-bf3e-850dee9d9cda None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:23.908214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b51e45-7867-4e54-bf3e-850dee9d9cda None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:23.909942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 120/243] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:54:23.918534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b375a0-82bb-4051-ad5c-78db2612265a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:23.926146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e67a8-fc5d-4960-b19a-0c165eb6e963 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:23.926618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e67a8-fc5d-4960-b19a-0c165eb6e963 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:23.927100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e67a8-fc5d-4960-b19a-0c165eb6e963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:23.974561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-071e67a8-fc5d-4960-b19a-0c165eb6e963 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:23.976724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-071e67a8-fc5d-4960-b19a-0c165eb6e963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:23.998631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6a8a7-1a63-40ca-b0b9-7974d2fc124a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:23.999635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6a8a7-1a63-40ca-b0b9-7974d2fc124a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:24.000829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6a8a7-1a63-40ca-b0b9-7974d2fc124a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:24.001152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6a8a7-1a63-40ca-b0b9-7974d2fc124a None admin] PATH_INFO: `/v3/users/31c1c71a5beb4639a267c3c1ce58a8ca` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:24.040154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6a8a7-1a63-40ca-b0b9-7974d2fc124a None admin] RBAC: Authorizing `identity:get_user(user_id=31c1c71a5beb4639a267c3c1ce58a8ca)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:24.043360 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6a8a7-1a63-40ca-b0b9-7974d2fc124a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:24.048730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 124/244] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:23 2022] GET /identity/v3/users/31c1c71a5beb4639a267c3c1ce58a8ca => generated 297 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:24.058100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6a8a7-1a63-40ca-b0b9-7974d2fc124a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:24.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 121/245] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:24.247245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:24.276781 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6a8a7-1a63-40ca-b0b9-7974d2fc124a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:24.322453 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19217ff3-1e99-4f25-a351-502ec22abb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:24.323188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19217ff3-1e99-4f25-a351-502ec22abb56 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:24.323412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19217ff3-1e99-4f25-a351-502ec22abb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:24.323655 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19217ff3-1e99-4f25-a351-502ec22abb56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:24.408718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19217ff3-1e99-4f25-a351-502ec22abb56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:24.410281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19217ff3-1e99-4f25-a351-502ec22abb56 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:24.513847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:24.537417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 125/246] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 20:54:24.545807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e67a8-fc5d-4960-b19a-0c165eb6e963 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:24.632573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19217ff3-1e99-4f25-a351-502ec22abb56 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:24.683257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e67a8-fc5d-4960-b19a-0c165eb6e963 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:24.713147 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:24.727399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2fdd7-3ab5-415c-a28b-bc07eb9b1cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:24.728062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2fdd7-3ab5-415c-a28b-bc07eb9b1cce None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:24.728343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2fdd7-3ab5-415c-a28b-bc07eb9b1cce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:24.728617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a2fdd7-3ab5-415c-a28b-bc07eb9b1cce None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:24.730108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2fdd7-3ab5-415c-a28b-bc07eb9b1cce None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:24.731688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a2fdd7-3ab5-415c-a28b-bc07eb9b1cce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:24.734602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 122/247] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:24 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:24.752750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2fdd7-3ab5-415c-a28b-bc07eb9b1cce None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:24.757438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19217ff3-1e99-4f25-a351-502ec22abb56 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:24.799433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8618ab90-b456-4b2f-ad97-0a5b69b512b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:24.800079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618ab90-b456-4b2f-ad97-0a5b69b512b2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:24.800364 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618ab90-b456-4b2f-ad97-0a5b69b512b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:24.800638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8618ab90-b456-4b2f-ad97-0a5b69b512b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:24.822455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8618ab90-b456-4b2f-ad97-0a5b69b512b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8618ab90-b456-4b2f-ad97-0a5b69b512b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:24.827376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:24.830018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:24.831894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:24.833335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 126/248] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:24.839924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bcad9-6ddf-4524-abae-43b3919ce6bf None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:24.842296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bcad9-6ddf-4524-abae-43b3919ce6bf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:24.842571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94bcad9-6ddf-4524-abae-43b3919ce6bf None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:24.844159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 127/249] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:24.845009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bcad9-6ddf-4524-abae-43b3919ce6bf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:24.920051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:24.960167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a2fdd7-3ab5-415c-a28b-bc07eb9b1cce None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:24.999101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94bcad9-6ddf-4524-abae-43b3919ce6bf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:25.001097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a2f34-8de0-4bf3-87c1-64ef58e465c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:25.001910 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a2f34-8de0-4bf3-87c1-64ef58e465c0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:25.002299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a2f34-8de0-4bf3-87c1-64ef58e465c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:25.002679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a2f34-8de0-4bf3-87c1-64ef58e465c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:25.003574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a2f34-8de0-4bf3-87c1-64ef58e465c0 None admin] RBAC: Adding query filter params (role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f, scope.project.id=4ff250f07e37455ba8616d6434ffbef6, user.id=31c1c71a5beb4639a267c3c1ce58a8ca) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:25.004003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a2f34-8de0-4bf3-87c1-64ef58e465c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:25.005721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a2f34-8de0-4bf3-87c1-64ef58e465c0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:25.012354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 123/250] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:24 2022] GET /identity/v3/role_assignments?user.id=31c1c71a5beb4639a267c3c1ce58a8ca&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:25.022642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25761f8e-1f34-4147-972e-15838efd0c82 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:25.023085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25761f8e-1f34-4147-972e-15838efd0c82 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:25.023536 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25761f8e-1f34-4147-972e-15838efd0c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:25.050364 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e70b1c-fb97-440f-8c03-cec602454a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:25.051376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e70b1c-fb97-440f-8c03-cec602454a27 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:25.051791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e70b1c-fb97-440f-8c03-cec602454a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:25.052245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e70b1c-fb97-440f-8c03-cec602454a27 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:25.053097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e70b1c-fb97-440f-8c03-cec602454a27 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:25.053561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e70b1c-fb97-440f-8c03-cec602454a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:25.055185 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e70b1c-fb97-440f-8c03-cec602454a27 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:25.066609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 128/251] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:25.067901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-25761f8e-1f34-4147-972e-15838efd0c82 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:25.068302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e294a1-855a-4f29-a718-3e73d524af90 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:25.068754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e294a1-855a-4f29-a718-3e73d524af90 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:25.069169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e294a1-855a-4f29-a718-3e73d524af90 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:25.069278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-25761f8e-1f34-4147-972e-15838efd0c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:25.071178 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 129/252] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:25.078010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a927-32f5-4d2b-b7e3-f8adec3077e4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:25.078599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a927-32f5-4d2b-b7e3-f8adec3077e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:25.079019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2a927-32f5-4d2b-b7e3-f8adec3077e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:25.114038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-87a2a927-32f5-4d2b-b7e3-f8adec3077e4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:25.115925 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-87a2a927-32f5-4d2b-b7e3-f8adec3077e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:25.199915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 130/253] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:25.205794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2a927-32f5-4d2b-b7e3-f8adec3077e4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:25.211154 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:25.257795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 124/254] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:25.284539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:25.367679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2a927-32f5-4d2b-b7e3-f8adec3077e4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:25.418646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7d62c-6030-4466-ac40-12783029671a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:25.419302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7d62c-6030-4466-ac40-12783029671a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:25.419583 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7d62c-6030-4466-ac40-12783029671a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:25.419864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e7d62c-6030-4466-ac40-12783029671a None admin] PATH_INFO: `/v3/users/4a01f1bb14ad4a0dbc5250126de78ece` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:25.426651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7d62c-6030-4466-ac40-12783029671a None admin] RBAC: Authorizing `identity:get_user(user_id=4a01f1bb14ad4a0dbc5250126de78ece)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:25.428182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e7d62c-6030-4466-ac40-12783029671a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:25.430323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 131/255] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:25 2022] GET /identity/v3/users/4a01f1bb14ad4a0dbc5250126de78ece => generated 297 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:25.436937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25761f8e-1f34-4147-972e-15838efd0c82 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:25.669889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25761f8e-1f34-4147-972e-15838efd0c82 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:25.691266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7d62c-6030-4466-ac40-12783029671a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:25.710425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1a320-e38f-4642-ad3a-495ff816a2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:25.711081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1a320-e38f-4642-ad3a-495ff816a2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:25.711361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1a320-e38f-4642-ad3a-495ff816a2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:25.711637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1a320-e38f-4642-ad3a-495ff816a2a7 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:25.712841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1a320-e38f-4642-ad3a-495ff816a2a7 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:25.714571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1a320-e38f-4642-ad3a-495ff816a2a7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:25.716628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 125/256] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:25 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:25.725187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1a320-e38f-4642-ad3a-495ff816a2a7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:25.810892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:25.888290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1a320-e38f-4642-ad3a-495ff816a2a7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:25.916383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e7d62c-6030-4466-ac40-12783029671a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:25.942288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:25.957201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20e719-e8d0-4991-99f4-4352dc54430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:25.958274 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20e719-e8d0-4991-99f4-4352dc54430b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:25.958750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20e719-e8d0-4991-99f4-4352dc54430b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:25.958977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904cbe5-422a-4e2a-aad0-22014a8cfa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:25.959471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20e719-e8d0-4991-99f4-4352dc54430b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:25.960139 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904cbe5-422a-4e2a-aad0-22014a8cfa76 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:25.960631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904cbe5-422a-4e2a-aad0-22014a8cfa76 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:25.961137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904cbe5-422a-4e2a-aad0-22014a8cfa76 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6/users/4a01f1bb14ad4a0dbc5250126de78ece/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:25.971314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904cbe5-422a-4e2a-aad0-22014a8cfa76 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ff250f07e37455ba8616d6434ffbef6, user_id=4a01f1bb14ad4a0dbc5250126de78ece, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:25.973856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b904cbe5-422a-4e2a-aad0-22014a8cfa76 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:25.974909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20e719-e8d0-4991-99f4-4352dc54430b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3b20e719-e8d0-4991-99f4-4352dc54430b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:25.980099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:25.981026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:25.981814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:25.985199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 132/257] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:25.986495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20e719-e8d0-4991-99f4-4352dc54430b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:26.005561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 126/258] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:54:25 2022] PUT /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6/users/4a01f1bb14ad4a0dbc5250126de78ece/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 20:54:26.006783 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904cbe5-422a-4e2a-aad0-22014a8cfa76 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:26.221226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904cbe5-422a-4e2a-aad0-22014a8cfa76 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:26.229412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20e719-e8d0-4991-99f4-4352dc54430b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:26.253095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:26.300720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a93eb5-ceba-4b84-9318-4573569c5c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:26.301361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a93eb5-ceba-4b84-9318-4573569c5c72 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:26.301638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a93eb5-ceba-4b84-9318-4573569c5c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:26.301940 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a93eb5-ceba-4b84-9318-4573569c5c72 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:26.302632 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a93eb5-ceba-4b84-9318-4573569c5c72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:26.302938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a93eb5-ceba-4b84-9318-4573569c5c72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:26.303847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa6743-48f4-4219-a8d2-87433b431d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:26.304487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa6743-48f4-4219-a8d2-87433b431d07 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:26.304753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa6743-48f4-4219-a8d2-87433b431d07 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:26.305016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa6743-48f4-4219-a8d2-87433b431d07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:26.307141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a93eb5-ceba-4b84-9318-4573569c5c72 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:26.324660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 127/259] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:26.328917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa6743-48f4-4219-a8d2-87433b431d07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:26.331208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a93eb5-ceba-4b84-9318-4573569c5c72 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0baa6743-48f4-4219-a8d2-87433b431d07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:26.335948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:26.336865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:26.337733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:26.340568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 133/260] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:54:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:26.354478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa6743-48f4-4219-a8d2-87433b431d07 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:26.401285 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:26.544752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa6743-48f4-4219-a8d2-87433b431d07 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:26.596028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a93eb5-ceba-4b84-9318-4573569c5c72 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:26.615350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762c694-08c6-448f-ada7-e3d507eb83c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:26.615984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762c694-08c6-448f-ada7-e3d507eb83c7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:26.616261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762c694-08c6-448f-ada7-e3d507eb83c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:26.616530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762c694-08c6-448f-ada7-e3d507eb83c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:26.617978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762c694-08c6-448f-ada7-e3d507eb83c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:26.618539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762c694-08c6-448f-ada7-e3d507eb83c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:26.620431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762c694-08c6-448f-ada7-e3d507eb83c7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:26.637192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 134/261] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:26.644367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762c694-08c6-448f-ada7-e3d507eb83c7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:26.690112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1d162-5fe5-4f86-93cd-142162ea65af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:26.691143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1d162-5fe5-4f86-93cd-142162ea65af None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:26.691923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1d162-5fe5-4f86-93cd-142162ea65af None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:26.692421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1d162-5fe5-4f86-93cd-142162ea65af None admin] PATH_INFO: `/v3/users/541c746bdc1a458b9036557defa4e094` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:26.695221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1d162-5fe5-4f86-93cd-142162ea65af None admin] RBAC: Authorizing `identity:get_user(user_id=541c746bdc1a458b9036557defa4e094)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:26.696988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb1d162-5fe5-4f86-93cd-142162ea65af None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:26.706555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 128/262] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:26 2022] GET /identity/v3/users/541c746bdc1a458b9036557defa4e094 => generated 283 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:26.709836 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1d162-5fe5-4f86-93cd-142162ea65af None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:26.821854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:26.862821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:26.894891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762c694-08c6-448f-ada7-e3d507eb83c7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:26.956641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb1d162-5fe5-4f86-93cd-142162ea65af None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:27.011667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3abcc6-82c3-4ee0-b1dd-bc1041389be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:27.013187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3abcc6-82c3-4ee0-b1dd-bc1041389be4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:27.014387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3abcc6-82c3-4ee0-b1dd-bc1041389be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:27.015225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3abcc6-82c3-4ee0-b1dd-bc1041389be4 None admin] PATH_INFO: `/v3/users/541c746bdc1a458b9036557defa4e094` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:27.018688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3abcc6-82c3-4ee0-b1dd-bc1041389be4 None admin] RBAC: Authorizing `identity:get_user(user_id=541c746bdc1a458b9036557defa4e094)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:27.020124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3abcc6-82c3-4ee0-b1dd-bc1041389be4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:27.022585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 135/263] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:26 2022] GET /identity/v3/users/541c746bdc1a458b9036557defa4e094 => generated 283 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:27.028374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3abcc6-82c3-4ee0-b1dd-bc1041389be4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:27.057282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6b6c5-a649-426b-acc5-b8f56dfddf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:27.057951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6b6c5-a649-426b-acc5-b8f56dfddf47 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:27.058235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6b6c5-a649-426b-acc5-b8f56dfddf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:27.058501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6b6c5-a649-426b-acc5-b8f56dfddf47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:27.171352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6b6c5-a649-426b-acc5-b8f56dfddf47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:27.172843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6b6c5-a649-426b-acc5-b8f56dfddf47 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:27.223095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:27.287230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 129/264] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:27.290301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3abcc6-82c3-4ee0-b1dd-bc1041389be4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:27.294659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6b6c5-a649-426b-acc5-b8f56dfddf47 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:27.339680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68917c29-9163-4174-86ed-1bf9fd66a915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:27.340203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68917c29-9163-4174-86ed-1bf9fd66a915 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:27.340478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68917c29-9163-4174-86ed-1bf9fd66a915 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:27.340757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68917c29-9163-4174-86ed-1bf9fd66a915 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:27.398477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:27.489712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68917c29-9163-4174-86ed-1bf9fd66a915 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:27.492034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68917c29-9163-4174-86ed-1bf9fd66a915 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:27.575826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6b6c5-a649-426b-acc5-b8f56dfddf47 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:27.645707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96587d-21d1-4b67-a566-98ebce824081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:27.647302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96587d-21d1-4b67-a566-98ebce824081 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:27.648086 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96587d-21d1-4b67-a566-98ebce824081 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:27.648241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 136/265] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:27.653855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d9cea-acb9-4094-b16d-ecb3fb7a4828 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:27.653855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d9cea-acb9-4094-b16d-ecb3fb7a4828 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:27.653855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0d9cea-acb9-4094-b16d-ecb3fb7a4828 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:27.653855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 137/266] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:27.653855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47160c15-b4ca-4a66-9e12-0da4d21b0e7d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:27.653855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47160c15-b4ca-4a66-9e12-0da4d21b0e7d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:27.654604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47160c15-b4ca-4a66-9e12-0da4d21b0e7d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:27.655495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96587d-21d1-4b67-a566-98ebce824081 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:27.658926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 138/267] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:27.660527 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96587d-21d1-4b67-a566-98ebce824081 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:27.660723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9608f-57c4-47c8-8011-60efcf15e579 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:27.660980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9608f-57c4-47c8-8011-60efcf15e579 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:27.661309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9608f-57c4-47c8-8011-60efcf15e579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:27.664253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c96587d-21d1-4b67-a566-98ebce824081 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:27.669982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 130/268] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:27 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:27.674618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96587d-21d1-4b67-a566-98ebce824081 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:27.693558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:27.698852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-96c9608f-57c4-47c8-8011-60efcf15e579 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:27.701481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-96c9608f-57c4-47c8-8011-60efcf15e579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:27.802171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96587d-21d1-4b67-a566-98ebce824081 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:27.802611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:27.833347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 139/269] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:27.836136 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fba955-fad1-4159-a0a3-b7e088d82980 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:27.836963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fba955-fad1-4159-a0a3-b7e088d82980 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:27.837757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fba955-fad1-4159-a0a3-b7e088d82980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:27.873212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48521a-047b-4b4c-a218-0bd88aa71a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:27.875610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48521a-047b-4b4c-a218-0bd88aa71a3d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:27.879040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48521a-047b-4b4c-a218-0bd88aa71a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:27.879894 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48521a-047b-4b4c-a218-0bd88aa71a3d None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:27.883474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48521a-047b-4b4c-a218-0bd88aa71a3d None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:27.892619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b48521a-047b-4b4c-a218-0bd88aa71a3d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:27.896429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 131/270] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:27 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:27.897875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e0fba955-fad1-4159-a0a3-b7e088d82980 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:27.900083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e0fba955-fad1-4159-a0a3-b7e088d82980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:27.901973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48521a-047b-4b4c-a218-0bd88aa71a3d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:28.078281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 140/271] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:28.084412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fba955-fad1-4159-a0a3-b7e088d82980 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:28.204117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:28.233856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48521a-047b-4b4c-a218-0bd88aa71a3d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:28.281863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fba955-fad1-4159-a0a3-b7e088d82980 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:28.310264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aede854-a070-4f2b-8b8b-8f51bf15d5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:28.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aede854-a070-4f2b-8b8b-8f51bf15d5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:28.311173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aede854-a070-4f2b-8b8b-8f51bf15d5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:28.311462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aede854-a070-4f2b-8b8b-8f51bf15d5f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:28.312856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aede854-a070-4f2b-8b8b-8f51bf15d5f4 None admin] RBAC: Adding query filter params (role.id=a631c57af3494b9c955799e0488518ae, scope.project.id=4ff250f07e37455ba8616d6434ffbef6, user.id=541c746bdc1a458b9036557defa4e094) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:28.313129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aede854-a070-4f2b-8b8b-8f51bf15d5f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:28.315244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:28.317929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aede854-a070-4f2b-8b8b-8f51bf15d5f4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:28.335173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 132/272] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:27 2022] GET /identity/v3/role_assignments?user.id=541c746bdc1a458b9036557defa4e094&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=a631c57af3494b9c955799e0488518ae => generated 608 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:28.337949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090f5c1-d919-4553-94d3-aec54eba7a1d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:28.338770 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090f5c1-d919-4553-94d3-aec54eba7a1d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:28.339528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090f5c1-d919-4553-94d3-aec54eba7a1d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:28.342042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 133/273] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:28.344201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090f5c1-d919-4553-94d3-aec54eba7a1d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:28.367110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e029a-37c3-40d5-b189-023a6eb00d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:28.368961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e029a-37c3-40d5-b189-023a6eb00d59 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:28.369803 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e029a-37c3-40d5-b189-023a6eb00d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:28.370587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e029a-37c3-40d5-b189-023a6eb00d59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:28.372380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e029a-37c3-40d5-b189-023a6eb00d59 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=4ff250f07e37455ba8616d6434ffbef6, user.id=541c746bdc1a458b9036557defa4e094) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:28.373184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e029a-37c3-40d5-b189-023a6eb00d59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:28.376608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e029a-37c3-40d5-b189-023a6eb00d59 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:28.386555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 141/274] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:28 2022] GET /identity/v3/role_assignments?user.id=541c746bdc1a458b9036557defa4e094&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=8b350845f331423cb0494e3449b58310 => generated 608 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:28.388771 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e871a-3bd1-495c-8449-489fe6bf440b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:28.389497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e871a-3bd1-495c-8449-489fe6bf440b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:28.390104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e871a-3bd1-495c-8449-489fe6bf440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:28.419827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2b4e871a-3bd1-495c-8449-489fe6bf440b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:28.422901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2b4e871a-3bd1-495c-8449-489fe6bf440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:28.585158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9090f5c1-d919-4553-94d3-aec54eba7a1d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:28.587862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 142/275] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:28.595592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e871a-3bd1-495c-8449-489fe6bf440b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:28.699038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a6a61-a85d-46cc-948d-273f30ea89f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:28.700546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a6a61-a85d-46cc-948d-273f30ea89f2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:28.701275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a6a61-a85d-46cc-948d-273f30ea89f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:28.702148 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a6a61-a85d-46cc-948d-273f30ea89f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:28.726398 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a6a61-a85d-46cc-948d-273f30ea89f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a10a6a61-a85d-46cc-948d-273f30ea89f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:28.740922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:28.742679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:28.744457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:28.748997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 134/276] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 406 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:28.758902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a6a61-a85d-46cc-948d-273f30ea89f2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:28.763668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:28.800876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e871a-3bd1-495c-8449-489fe6bf440b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:28.816206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:28.856000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3169263-3f9a-40a1-8cde-e9ecf3b58a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:28.857071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3169263-3f9a-40a1-8cde-e9ecf3b58a06 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:28.857245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3169263-3f9a-40a1-8cde-e9ecf3b58a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:28.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3169263-3f9a-40a1-8cde-e9ecf3b58a06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:28.860118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3169263-3f9a-40a1-8cde-e9ecf3b58a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:28.861500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3169263-3f9a-40a1-8cde-e9ecf3b58a06 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:28.865681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 143/277] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:54:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:28.872258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3169263-3f9a-40a1-8cde-e9ecf3b58a06 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:28.984276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a6a61-a85d-46cc-948d-273f30ea89f2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:29.048453 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3169263-3f9a-40a1-8cde-e9ecf3b58a06 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:29.049344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c0dbe-980a-4e47-9966-4613819e09e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:29.050482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c0dbe-980a-4e47-9966-4613819e09e9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:29.050980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c0dbe-980a-4e47-9966-4613819e09e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:29.051570 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c0dbe-980a-4e47-9966-4613819e09e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:29.053585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c0dbe-980a-4e47-9966-4613819e09e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:29.053585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c0dbe-980a-4e47-9966-4613819e09e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:29.056180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6c0dbe-980a-4e47-9966-4613819e09e9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:29.075475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 135/278] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:29.095874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c0dbe-980a-4e47-9966-4613819e09e9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:29.119246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a4f9-2635-427e-9bc5-5471fd7ea268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:29.119764 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a4f9-2635-427e-9bc5-5471fd7ea268 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:29.120030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a4f9-2635-427e-9bc5-5471fd7ea268 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:29.120309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1a4f9-2635-427e-9bc5-5471fd7ea268 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:29.121421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a4f9-2635-427e-9bc5-5471fd7ea268 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:29.123160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1a4f9-2635-427e-9bc5-5471fd7ea268 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:29.158416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 144/279] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:54:28 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:54:29.228019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a4f9-2635-427e-9bc5-5471fd7ea268 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:29.240662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:29.304865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:29.315414 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c0dbe-980a-4e47-9966-4613819e09e9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:29.371470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126ab449-0260-44bc-826e-eb092ad341d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:29.371470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ab449-0260-44bc-826e-eb092ad341d9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:29.371470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ab449-0260-44bc-826e-eb092ad341d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:29.371470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126ab449-0260-44bc-826e-eb092ad341d9 None admin] PATH_INFO: `/v3/users/e7362d182871460ca31055bd28f62fcd` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:29.393577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ab449-0260-44bc-826e-eb092ad341d9 None admin] RBAC: Authorizing `identity:get_user(user_id=e7362d182871460ca31055bd28f62fcd)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:29.395551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126ab449-0260-44bc-826e-eb092ad341d9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:29.398067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 136/280] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:29 2022] GET /identity/v3/users/e7362d182871460ca31055bd28f62fcd => generated 293 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:29.399892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501b519-d9a1-4bd5-af52-7071c51f2b33 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:29.400960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501b519-d9a1-4bd5-af52-7071c51f2b33 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:29.400960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501b519-d9a1-4bd5-af52-7071c51f2b33 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:29.403489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 137/281] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:29.410358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501b519-d9a1-4bd5-af52-7071c51f2b33 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:29.428438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1a4f9-2635-427e-9bc5-5471fd7ea268 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:29.485645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077b4ca-60cf-48c4-8af3-142ff71735cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:29.486930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077b4ca-60cf-48c4-8af3-142ff71735cb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:29.487431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077b4ca-60cf-48c4-8af3-142ff71735cb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:29.488218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077b4ca-60cf-48c4-8af3-142ff71735cb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:29.508257 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077b4ca-60cf-48c4-8af3-142ff71735cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2077b4ca-60cf-48c4-8af3-142ff71735cb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:29.519571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:29.520516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:29.521332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:29.525954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 145/282] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:54:29.528372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044f753-3221-4b72-bb1e-6b0b75f5cf0d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:29.529703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044f753-3221-4b72-bb1e-6b0b75f5cf0d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:29.531412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044f753-3221-4b72-bb1e-6b0b75f5cf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:29.565145 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0044f753-3221-4b72-bb1e-6b0b75f5cf0d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:29.568166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0044f753-3221-4b72-bb1e-6b0b75f5cf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:29.576039 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501b519-d9a1-4bd5-af52-7071c51f2b33 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:29.609803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:29.630099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ccd0d-17c8-4e96-aa76-c4ea4f5d84b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:29.631061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ccd0d-17c8-4e96-aa76-c4ea4f5d84b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:29.631543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ccd0d-17c8-4e96-aa76-c4ea4f5d84b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:29.632056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ccd0d-17c8-4e96-aa76-c4ea4f5d84b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:29.633036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ccd0d-17c8-4e96-aa76-c4ea4f5d84b3 None admin] RBAC: Adding query filter params (role.id=f8c927860d6941559283c83fcaaf93d8, user.id=e7362d182871460ca31055bd28f62fcd, scope.system=all) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:29.633529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ccd0d-17c8-4e96-aa76-c4ea4f5d84b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:29.635071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ccd0d-17c8-4e96-aa76-c4ea4f5d84b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:29.644966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 138/283] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Aug 19 20:54:29 2022] GET /identity/v3/role_assignments?user.id=e7362d182871460ca31055bd28f62fcd&scope.system=all&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:29.645600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ccd0d-17c8-4e96-aa76-c4ea4f5d84b3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:29.769753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 146/284] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:29.856965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ccd0d-17c8-4e96-aa76-c4ea4f5d84b3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:29.926447 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3a1fb-b4b6-4895-9f0b-304f3a3181f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:29.927418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3a1fb-b4b6-4895-9f0b-304f3a3181f9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:29.927925 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3a1fb-b4b6-4895-9f0b-304f3a3181f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:29.928417 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd3a1fb-b4b6-4895-9f0b-304f3a3181f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:29.929627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3a1fb-b4b6-4895-9f0b-304f3a3181f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:29.930193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3a1fb-b4b6-4895-9f0b-304f3a3181f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:29.932825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd3a1fb-b4b6-4895-9f0b-304f3a3181f9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:29.950307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 139/285] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:29.957919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044f753-3221-4b72-bb1e-6b0b75f5cf0d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:30.010972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:30.212948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044f753-3221-4b72-bb1e-6b0b75f5cf0d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:30.281427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff05b8-6022-4f5d-8480-0809100ec8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:30.282580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff05b8-6022-4f5d-8480-0809100ec8ee None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:30.283090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff05b8-6022-4f5d-8480-0809100ec8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:30.283596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff05b8-6022-4f5d-8480-0809100ec8ee None admin] PATH_INFO: `/v3/users/31c1c71a5beb4639a267c3c1ce58a8ca` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:30.285664 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff05b8-6022-4f5d-8480-0809100ec8ee None admin] RBAC: Authorizing `identity:get_user(user_id=31c1c71a5beb4639a267c3c1ce58a8ca)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:30.288541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff05b8-6022-4f5d-8480-0809100ec8ee None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:30.292146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 147/286] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:29 2022] GET /identity/v3/users/31c1c71a5beb4639a267c3c1ce58a8ca => generated 297 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:30.301713 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3a1fb-b4b6-4895-9f0b-304f3a3181f9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:30.414517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:30.430248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff05b8-6022-4f5d-8480-0809100ec8ee None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:30.539689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd3a1fb-b4b6-4895-9f0b-304f3a3181f9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:30.614115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbb5a8-1168-4d7e-acc5-20172b23b1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:30.614781 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbb5a8-1168-4d7e-acc5-20172b23b1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:30.615065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbb5a8-1168-4d7e-acc5-20172b23b1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:30.615369 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbb5a8-1168-4d7e-acc5-20172b23b1c5 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:30.616611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cbb5a8-1168-4d7e-acc5-20172b23b1c5 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:30.618120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cbb5a8-1168-4d7e-acc5-20172b23b1c5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:30.620664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 140/287] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:30 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:30.629723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbb5a8-1168-4d7e-acc5-20172b23b1c5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:30.661973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:30.680644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff05b8-6022-4f5d-8480-0809100ec8ee None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:30.750469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0ae24-bc47-42f5-ad07-73a3dd8b6735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:30.751134 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0ae24-bc47-42f5-ad07-73a3dd8b6735 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:30.751409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0ae24-bc47-42f5-ad07-73a3dd8b6735 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:30.751686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0ae24-bc47-42f5-ad07-73a3dd8b6735 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:30.774064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f0ae24-bc47-42f5-ad07-73a3dd8b6735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-00f0ae24-bc47-42f5-ad07-73a3dd8b6735 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:30.786406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:30.788031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:30.789510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:30.802245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 148/288] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:54:30.804194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0ae24-bc47-42f5-ad07-73a3dd8b6735 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:30.819938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cbb5a8-1168-4d7e-acc5-20172b23b1c5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:30.842323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:30.863174 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe78a8f-0e2b-470c-9898-ad3eb2edc37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:30.863855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe78a8f-0e2b-470c-9898-ad3eb2edc37d None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:30.864138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe78a8f-0e2b-470c-9898-ad3eb2edc37d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:30.864420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe78a8f-0e2b-470c-9898-ad3eb2edc37d None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6/users/31c1c71a5beb4639a267c3c1ce58a8ca/roles/4bbefa34d6764a6fa5b84ff6d1a4f09f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:30.875620 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe78a8f-0e2b-470c-9898-ad3eb2edc37d None admin] RBAC: Authorizing `identity:create_grant(project_id=4ff250f07e37455ba8616d6434ffbef6, user_id=31c1c71a5beb4639a267c3c1ce58a8ca, role_id=4bbefa34d6764a6fa5b84ff6d1a4f09f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:30.877122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe78a8f-0e2b-470c-9898-ad3eb2edc37d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:30.904202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 141/289] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:54:30 2022] PUT /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6/users/31c1c71a5beb4639a267c3c1ce58a8ca/roles/4bbefa34d6764a6fa5b84ff6d1a4f09f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 20:54:31.019228 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f0ae24-bc47-42f5-ad07-73a3dd8b6735 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:31.082492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55bb16-8042-431c-9b36-3e697ce6e587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:31.084000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55bb16-8042-431c-9b36-3e697ce6e587 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:31.084742 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55bb16-8042-431c-9b36-3e697ce6e587 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:31.085466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e55bb16-8042-431c-9b36-3e697ce6e587 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:31.087056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55bb16-8042-431c-9b36-3e697ce6e587 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:31.089792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55bb16-8042-431c-9b36-3e697ce6e587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:31.090194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e55bb16-8042-431c-9b36-3e697ce6e587 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:31.106465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 149/290] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:31.115907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe78a8f-0e2b-470c-9898-ad3eb2edc37d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:31.322053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe78a8f-0e2b-470c-9898-ad3eb2edc37d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:31.391899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaf36d-348d-4461-b044-72ab462e31be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:31.392912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaf36d-348d-4461-b044-72ab462e31be None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:31.393408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaf36d-348d-4461-b044-72ab462e31be None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:31.393921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eaf36d-348d-4461-b044-72ab462e31be None admin] PATH_INFO: `/v3/users/4a01f1bb14ad4a0dbc5250126de78ece` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:31.396146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eaf36d-348d-4461-b044-72ab462e31be None admin] RBAC: Authorizing `identity:get_user(user_id=4a01f1bb14ad4a0dbc5250126de78ece)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:31.399022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eaf36d-348d-4461-b044-72ab462e31be None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:31.402464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 142/291] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:31 2022] GET /identity/v3/users/4a01f1bb14ad4a0dbc5250126de78ece => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:31.411495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55bb16-8042-431c-9b36-3e697ce6e587 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:31.442651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:31.605557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e55bb16-8042-431c-9b36-3e697ce6e587 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:31.672326 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354de36a-6869-4068-9130-6ff3ecb3a489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:31.673268 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354de36a-6869-4068-9130-6ff3ecb3a489 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:31.673748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354de36a-6869-4068-9130-6ff3ecb3a489 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:31.674256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354de36a-6869-4068-9130-6ff3ecb3a489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:31.787246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354de36a-6869-4068-9130-6ff3ecb3a489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:31.788714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354de36a-6869-4068-9130-6ff3ecb3a489 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:31.811704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:31.941098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 150/292] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:31.946710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaf36d-348d-4461-b044-72ab462e31be None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:32.001110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:32.062492 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eaf36d-348d-4461-b044-72ab462e31be None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:32.100561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46708f36-862d-4112-a98a-64fe4742b930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:32.101297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46708f36-862d-4112-a98a-64fe4742b930 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:32.101577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46708f36-862d-4112-a98a-64fe4742b930 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:32.101876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46708f36-862d-4112-a98a-64fe4742b930 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:32.103036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46708f36-862d-4112-a98a-64fe4742b930 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:32.104586 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46708f36-862d-4112-a98a-64fe4742b930 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:32.106561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 143/293] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:31 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:32.113850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354de36a-6869-4068-9130-6ff3ecb3a489 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:32.322534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354de36a-6869-4068-9130-6ff3ecb3a489 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:32.359596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:32.393375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87896a93-ff09-4377-8250-5aa2ab7b2a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:32.394028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87896a93-ff09-4377-8250-5aa2ab7b2a3c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:32.394308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87896a93-ff09-4377-8250-5aa2ab7b2a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:32.394584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87896a93-ff09-4377-8250-5aa2ab7b2a3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:32.395334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87896a93-ff09-4377-8250-5aa2ab7b2a3c None admin] RBAC: Adding query filter params (role.id=f8c927860d6941559283c83fcaaf93d8, scope.project.id=4ff250f07e37455ba8616d6434ffbef6, user.id=4a01f1bb14ad4a0dbc5250126de78ece) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:32.395642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87896a93-ff09-4377-8250-5aa2ab7b2a3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:32.397118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87896a93-ff09-4377-8250-5aa2ab7b2a3c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:32.408652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 151/294] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:32 2022] GET /identity/v3/role_assignments?user.id=4a01f1bb14ad4a0dbc5250126de78ece&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:32.571023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fedb8-3223-4c0f-8ac0-b56396954d7b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:32.571784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fedb8-3223-4c0f-8ac0-b56396954d7b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:32.572134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fedb8-3223-4c0f-8ac0-b56396954d7b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:32.575079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 144/295] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:32.583140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac376fbf-6875-45d8-ab71-77c03d40d1ce None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:32.583534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac376fbf-6875-45d8-ab71-77c03d40d1ce None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:32.583749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac376fbf-6875-45d8-ab71-77c03d40d1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:32.602902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095948a-73a6-4e42-b265-b0a0b4df51fa None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:32.603299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095948a-73a6-4e42-b265-b0a0b4df51fa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:32.603668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6095948a-73a6-4e42-b265-b0a0b4df51fa None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:32.606601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 145/296] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:32.610317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ac376fbf-6875-45d8-ab71-77c03d40d1ce None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:32.611877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ac376fbf-6875-45d8-ab71-77c03d40d1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:32.621791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec0a9-99ef-48da-93d5-ed5ad6c927e8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:32.621791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec0a9-99ef-48da-93d5-ed5ad6c927e8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:32.621791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec0a9-99ef-48da-93d5-ed5ad6c927e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:32.658248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-957ec0a9-99ef-48da-93d5-ed5ad6c927e8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:32.660473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-957ec0a9-99ef-48da-93d5-ed5ad6c927e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:32.772406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 152/297] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:32.799224 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 146/298] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:33.120877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd7c70-67b1-4fd4-9696-133283c52e6a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:33.122396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd7c70-67b1-4fd4-9696-133283c52e6a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:33.122982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd7c70-67b1-4fd4-9696-133283c52e6a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:33.125474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 153/299] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:33.133990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0966e3-62f5-4a5b-901e-b85b860c6dad None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:33.134582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0966e3-62f5-4a5b-901e-b85b860c6dad None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:33.135055 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0966e3-62f5-4a5b-901e-b85b860c6dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:33.141841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd7c70-67b1-4fd4-9696-133283c52e6a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:33.161507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2f0966e3-62f5-4a5b-901e-b85b860c6dad None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:33.162785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2f0966e3-62f5-4a5b-901e-b85b860c6dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:33.215057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:33.260662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 147/300] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:33.261994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0966e3-62f5-4a5b-901e-b85b860c6dad None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:33.306457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd7c70-67b1-4fd4-9696-133283c52e6a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:33.324168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:33.377730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be045b09-ccd3-4a58-a861-48eef4d01528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:33.379136 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be045b09-ccd3-4a58-a861-48eef4d01528 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:33.379832 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be045b09-ccd3-4a58-a861-48eef4d01528 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:33.380718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be045b09-ccd3-4a58-a861-48eef4d01528 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:33.400976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be045b09-ccd3-4a58-a861-48eef4d01528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-be045b09-ccd3-4a58-a861-48eef4d01528 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:33.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:33.412748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:33.414603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:33.422365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 154/301] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:33.429131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be045b09-ccd3-4a58-a861-48eef4d01528 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:33.470312 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0966e3-62f5-4a5b-901e-b85b860c6dad None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:33.537334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1957a9-0929-465d-b7e0-35080abf0c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:33.538860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1957a9-0929-465d-b7e0-35080abf0c31 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:33.539630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1957a9-0929-465d-b7e0-35080abf0c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:33.540395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1957a9-0929-465d-b7e0-35080abf0c31 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:33.542876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1957a9-0929-465d-b7e0-35080abf0c31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:33.547005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1957a9-0929-465d-b7e0-35080abf0c31 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:33.551127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 148/302] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:54:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:33.560079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1957a9-0929-465d-b7e0-35080abf0c31 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:33.630448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:33.634451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be045b09-ccd3-4a58-a861-48eef4d01528 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:33.674342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934af42d-a1fe-4f67-aa27-d48d06303533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:33.674963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934af42d-a1fe-4f67-aa27-d48d06303533 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:33.675234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934af42d-a1fe-4f67-aa27-d48d06303533 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:33.675507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934af42d-a1fe-4f67-aa27-d48d06303533 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:33.676185 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934af42d-a1fe-4f67-aa27-d48d06303533 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:33.676486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934af42d-a1fe-4f67-aa27-d48d06303533 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:33.677914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934af42d-a1fe-4f67-aa27-d48d06303533 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:33.692471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 155/303] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:33.692471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97e128-8831-4b51-b1fe-604b65dd3dec None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:33.692471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97e128-8831-4b51-b1fe-604b65dd3dec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:33.692471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97e128-8831-4b51-b1fe-604b65dd3dec None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:33.692471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 156/304] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:33.692471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97e128-8831-4b51-b1fe-604b65dd3dec None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:33.725479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:33.729201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1957a9-0929-465d-b7e0-35080abf0c31 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:33.778544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437426c9-67cb-45d9-8911-e2b97df1a672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:33.779517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437426c9-67cb-45d9-8911-e2b97df1a672 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:33.779907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437426c9-67cb-45d9-8911-e2b97df1a672 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:33.781003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437426c9-67cb-45d9-8911-e2b97df1a672 None admin] PATH_INFO: `/v3/groups` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:33.782202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437426c9-67cb-45d9-8911-e2b97df1a672 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:33.785096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437426c9-67cb-45d9-8911-e2b97df1a672 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:33.810601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 149/305] 10.222.0.16 () {60 vars in 1134 bytes} [Fri Aug 19 20:54:33 2022] POST /identity/v3/groups => generated 229 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:54:33.812229 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437426c9-67cb-45d9-8911-e2b97df1a672 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:33.815394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97e128-8831-4b51-b1fe-604b65dd3dec None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:33.869374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550102de-fbde-4d21-b1a8-f649509ee60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:33.870263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550102de-fbde-4d21-b1a8-f649509ee60f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:33.870709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550102de-fbde-4d21-b1a8-f649509ee60f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:33.871341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550102de-fbde-4d21-b1a8-f649509ee60f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:33.888770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550102de-fbde-4d21-b1a8-f649509ee60f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-550102de-fbde-4d21-b1a8-f649509ee60f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:33.895341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:33.896618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 157/306] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:33.898265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8187a-2315-45a0-92c7-5fd3beb5d9af None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:33.899671 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8187a-2315-45a0-92c7-5fd3beb5d9af None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:33.899671 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8187a-2315-45a0-92c7-5fd3beb5d9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:33.920375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7fb8187a-2315-45a0-92c7-5fd3beb5d9af None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:33.922266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7fb8187a-2315-45a0-92c7-5fd3beb5d9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:33.981272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437426c9-67cb-45d9-8911-e2b97df1a672 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:33.985533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:34.005235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 158/307] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:34.005975 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8187a-2315-45a0-92c7-5fd3beb5d9af None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:34.039014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:34.070947 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aedfd7-5e5e-461e-988e-e0a920f9d9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:34.071612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aedfd7-5e5e-461e-988e-e0a920f9d9a3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:34.071901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aedfd7-5e5e-461e-988e-e0a920f9d9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:34.072169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aedfd7-5e5e-461e-988e-e0a920f9d9a3 None admin] PATH_INFO: `/v3/users/a72ecaf8da434213a7bad5df8d20eedf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:34.088746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aedfd7-5e5e-461e-988e-e0a920f9d9a3 None admin] RBAC: Authorizing `identity:get_user(user_id=a72ecaf8da434213a7bad5df8d20eedf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:34.090307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aedfd7-5e5e-461e-988e-e0a920f9d9a3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:34.092102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 150/308] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:33 2022] GET /identity/v3/users/a72ecaf8da434213a7bad5df8d20eedf => generated 293 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:34.102812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aedfd7-5e5e-461e-988e-e0a920f9d9a3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:34.236805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8187a-2315-45a0-92c7-5fd3beb5d9af None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:34.310136 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9992e40-a07b-4668-ab46-7de2a67642ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:34.311393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9992e40-a07b-4668-ab46-7de2a67642ae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:34.312581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9992e40-a07b-4668-ab46-7de2a67642ae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:34.313132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9992e40-a07b-4668-ab46-7de2a67642ae None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:34.314556 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9992e40-a07b-4668-ab46-7de2a67642ae None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:34.315054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9992e40-a07b-4668-ab46-7de2a67642ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:34.317626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9992e40-a07b-4668-ab46-7de2a67642ae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:34.328033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aedfd7-5e5e-461e-988e-e0a920f9d9a3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:34.334293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 159/309] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:34.339209 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9992e40-a07b-4668-ab46-7de2a67642ae None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:34.402664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed76a7-7f2c-4d97-a38f-a9740a311031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:34.403709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed76a7-7f2c-4d97-a38f-a9740a311031 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:34.404208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed76a7-7f2c-4d97-a38f-a9740a311031 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:34.404714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ed76a7-7f2c-4d97-a38f-a9740a311031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:34.406186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed76a7-7f2c-4d97-a38f-a9740a311031 None admin] RBAC: Adding query filter params (role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f, user.id=a72ecaf8da434213a7bad5df8d20eedf, scope.system=all) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:34.406740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed76a7-7f2c-4d97-a38f-a9740a311031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:34.409521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ed76a7-7f2c-4d97-a38f-a9740a311031 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:34.419534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 151/310] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Aug 19 20:54:34 2022] GET /identity/v3/role_assignments?user.id=a72ecaf8da434213a7bad5df8d20eedf&scope.system=all&role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f => generated 226 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:34.421716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed76a7-7f2c-4d97-a38f-a9740a311031 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:34.472820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:34.482296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:34.495990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9992e40-a07b-4668-ab46-7de2a67642ae None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:34.537826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff45ae-0ee7-446b-9e86-8ea84829de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:34.538500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff45ae-0ee7-446b-9e86-8ea84829de4f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:34.538779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff45ae-0ee7-446b-9e86-8ea84829de4f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:34.539055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff45ae-0ee7-446b-9e86-8ea84829de4f None admin] PATH_INFO: `/v3/users/e7362d182871460ca31055bd28f62fcd` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:34.540376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff45ae-0ee7-446b-9e86-8ea84829de4f None admin] RBAC: Authorizing `identity:get_user(user_id=e7362d182871460ca31055bd28f62fcd)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:34.541923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff45ae-0ee7-446b-9e86-8ea84829de4f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:34.545801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 160/311] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:34 2022] GET /identity/v3/users/e7362d182871460ca31055bd28f62fcd => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:34.553541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff45ae-0ee7-446b-9e86-8ea84829de4f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:34.633692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ed76a7-7f2c-4d97-a38f-a9740a311031 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:34.703261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815a59a-047e-4b1a-a3e9-1d672c607b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:34.704516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815a59a-047e-4b1a-a3e9-1d672c607b35 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:34.705180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815a59a-047e-4b1a-a3e9-1d672c607b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:34.705685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815a59a-047e-4b1a-a3e9-1d672c607b35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:34.712837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff45ae-0ee7-446b-9e86-8ea84829de4f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:34.727835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815a59a-047e-4b1a-a3e9-1d672c607b35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b815a59a-047e-4b1a-a3e9-1d672c607b35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:34.740051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:34.740998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:34.741784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:34.742399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 152/312] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:34.748901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815a59a-047e-4b1a-a3e9-1d672c607b35 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:34.759424 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e2841-f094-4b6c-82b5-0c7e3917b604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:34.760091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e2841-f094-4b6c-82b5-0c7e3917b604 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:34.760367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e2841-f094-4b6c-82b5-0c7e3917b604 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:34.760641 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38e2841-f094-4b6c-82b5-0c7e3917b604 None admin] PATH_INFO: `/v3/system/users/e7362d182871460ca31055bd28f62fcd/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:34.762214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e2841-f094-4b6c-82b5-0c7e3917b604 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e7362d182871460ca31055bd28f62fcd, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:34.763908 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38e2841-f094-4b6c-82b5-0c7e3917b604 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:34.777956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 161/313] 10.222.0.16 () {58 vars in 1334 bytes} [Fri Aug 19 20:54:34 2022] PUT /identity/v3/system/users/e7362d182871460ca31055bd28f62fcd/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 20:54:34.861419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:34.881485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815a59a-047e-4b1a-a3e9-1d672c607b35 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:34.928681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2f1f4-91ea-4f77-933d-98090d936753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:34.929314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2f1f4-91ea-4f77-933d-98090d936753 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:34.929593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2f1f4-91ea-4f77-933d-98090d936753 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:34.929898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2f1f4-91ea-4f77-933d-98090d936753 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:34.930593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2f1f4-91ea-4f77-933d-98090d936753 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:34.930897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2f1f4-91ea-4f77-933d-98090d936753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:34.932334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2f1f4-91ea-4f77-933d-98090d936753 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:34.944227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 153/314] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:34.950974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e2841-f094-4b6c-82b5-0c7e3917b604 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:34.970458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:35.074395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38e2841-f094-4b6c-82b5-0c7e3917b604 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:35.116398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea57c67-9805-430a-900b-b119152fc5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:35.117125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea57c67-9805-430a-900b-b119152fc5bb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:35.117402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea57c67-9805-430a-900b-b119152fc5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:35.117672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea57c67-9805-430a-900b-b119152fc5bb None admin] PATH_INFO: `/v3/users/31c1c71a5beb4639a267c3c1ce58a8ca` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:35.119275 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea57c67-9805-430a-900b-b119152fc5bb None admin] RBAC: Authorizing `identity:get_user(user_id=31c1c71a5beb4639a267c3c1ce58a8ca)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:35.120755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea57c67-9805-430a-900b-b119152fc5bb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:35.122783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 162/315] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:34 2022] GET /identity/v3/users/31c1c71a5beb4639a267c3c1ce58a8ca => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:35.128382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2f1f4-91ea-4f77-933d-98090d936753 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:35.258786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2f1f4-91ea-4f77-933d-98090d936753 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:35.298837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f463a-6051-4110-8aee-1718a7dc9193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:35.299934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f463a-6051-4110-8aee-1718a7dc9193 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:35.300421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f463a-6051-4110-8aee-1718a7dc9193 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:35.300923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f463a-6051-4110-8aee-1718a7dc9193 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:35.333282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:35.440186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f463a-6051-4110-8aee-1718a7dc9193 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:35.443216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f463a-6051-4110-8aee-1718a7dc9193 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:35.636760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 154/316] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:35.643530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea57c67-9805-430a-900b-b119152fc5bb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:35.776769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:35.806800 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea57c67-9805-430a-900b-b119152fc5bb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:35.846669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080418b-10cd-4d46-a86f-746de7d57df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:35.847329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080418b-10cd-4d46-a86f-746de7d57df0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:35.847612 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080418b-10cd-4d46-a86f-746de7d57df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:35.847885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080418b-10cd-4d46-a86f-746de7d57df0 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:35.849049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080418b-10cd-4d46-a86f-746de7d57df0 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:35.850548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080418b-10cd-4d46-a86f-746de7d57df0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:35.852553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 163/317] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:35 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:35.859658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f463a-6051-4110-8aee-1718a7dc9193 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:35.910922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:35.971814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f463a-6051-4110-8aee-1718a7dc9193 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:36.009543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc121310-9994-447b-9ffb-541a6de5a452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:36.010196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc121310-9994-447b-9ffb-541a6de5a452 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:36.010470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc121310-9994-447b-9ffb-541a6de5a452 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:36.010741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc121310-9994-447b-9ffb-541a6de5a452 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:36.011483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc121310-9994-447b-9ffb-541a6de5a452 None admin] RBAC: Adding query filter params (role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f, scope.project.id=4ff250f07e37455ba8616d6434ffbef6, user.id=31c1c71a5beb4639a267c3c1ce58a8ca) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:36.011789 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc121310-9994-447b-9ffb-541a6de5a452 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:36.013252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc121310-9994-447b-9ffb-541a6de5a452 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:36.021345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 155/318] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:54:35 2022] GET /identity/v3/role_assignments?user.id=31c1c71a5beb4639a267c3c1ce58a8ca&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:36.740579 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c82e-7d8a-4ef6-add5-7174f3065c1e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:36.741467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c82e-7d8a-4ef6-add5-7174f3065c1e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:36.741467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658c82e-7d8a-4ef6-add5-7174f3065c1e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:36.742719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 164/319] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:54:36.748128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab014a8-290c-4884-9052-1f995dfbe838 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:36.748813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab014a8-290c-4884-9052-1f995dfbe838 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:36.748813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab014a8-290c-4884-9052-1f995dfbe838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:36.768892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6ab014a8-290c-4884-9052-1f995dfbe838 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:36.770140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ab014a8-290c-4884-9052-1f995dfbe838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:36.849134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 156/320] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:36.981419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b048c-8cee-488e-82ba-05503c8c5763 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:36.981698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b048c-8cee-488e-82ba-05503c8c5763 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:36.982066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b048c-8cee-488e-82ba-05503c8c5763 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:36.983657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 165/321] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:54:36.988984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653d59-10cf-4faf-b329-1950c555822a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:36.989293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653d59-10cf-4faf-b329-1950c555822a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:36.989560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c653d59-10cf-4faf-b329-1950c555822a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:37.009803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0c653d59-10cf-4faf-b329-1950c555822a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:37.011025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0c653d59-10cf-4faf-b329-1950c555822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:37.058417 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:37.097822 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 157/322] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:37.122643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b048c-8cee-488e-82ba-05503c8c5763 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:37.201523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9320f-0f19-4f9d-afbf-9575c71e1f09 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:37.201864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9320f-0f19-4f9d-afbf-9575c71e1f09 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:37.202145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed9320f-0f19-4f9d-afbf-9575c71e1f09 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:37.203738 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 158/323] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:37.209130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb66ec-be52-4942-a1c3-8935a2a941c0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:37.209435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb66ec-be52-4942-a1c3-8935a2a941c0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:37.210383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb66ec-be52-4942-a1c3-8935a2a941c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:37.232663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fbbb66ec-be52-4942-a1c3-8935a2a941c0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:37.234219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fbbb66ec-be52-4942-a1c3-8935a2a941c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:37.243144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b048c-8cee-488e-82ba-05503c8c5763 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:37.287896 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cc234-f31c-4d10-a3c0-895754762cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:37.288456 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc234-f31c-4d10-a3c0-895754762cac None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:37.288798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc234-f31c-4d10-a3c0-895754762cac None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:37.289133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cc234-f31c-4d10-a3c0-895754762cac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:37.302738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748cc234-f31c-4d10-a3c0-895754762cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-748cc234-f31c-4d10-a3c0-895754762cac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:37.309007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:37.310895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:37.313013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 166/324] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:37.317637 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cc234-f31c-4d10-a3c0-895754762cac None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:37.332282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 159/325] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:37.341375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:37.393849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb66ec-be52-4942-a1c3-8935a2a941c0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:37.481162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cc234-f31c-4d10-a3c0-895754762cac None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:37.512085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:37.516968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb66ec-be52-4942-a1c3-8935a2a941c0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:37.524514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f8153-f8ed-4370-9363-8e642ced08e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:37.525134 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f8153-f8ed-4370-9363-8e642ced08e9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:37.525413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f8153-f8ed-4370-9363-8e642ced08e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:37.526247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f8153-f8ed-4370-9363-8e642ced08e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:37.526935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f8153-f8ed-4370-9363-8e642ced08e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:37.527239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f8153-f8ed-4370-9363-8e642ced08e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:37.529435 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f8153-f8ed-4370-9363-8e642ced08e9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:37.542472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 167/326] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:37.548688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f8153-f8ed-4370-9363-8e642ced08e9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:37.565225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b46496-ad5e-4ca6-9ea9-92e962c8f431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:37.565896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b46496-ad5e-4ca6-9ea9-92e962c8f431 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:37.566187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b46496-ad5e-4ca6-9ea9-92e962c8f431 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:37.566497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b46496-ad5e-4ca6-9ea9-92e962c8f431 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:37.579479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b46496-ad5e-4ca6-9ea9-92e962c8f431 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c3b46496-ad5e-4ca6-9ea9-92e962c8f431 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:37.584557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:37.586073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:37.587640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 160/327] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:54:37.593466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b46496-ad5e-4ca6-9ea9-92e962c8f431 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:37.648750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:37.705591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f8153-f8ed-4370-9363-8e642ced08e9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:37.714849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b46496-ad5e-4ca6-9ea9-92e962c8f431 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:37.755058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886867ed-ddfb-44ee-8fd5-96c3be13f616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:37.755700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886867ed-ddfb-44ee-8fd5-96c3be13f616 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:37.755990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886867ed-ddfb-44ee-8fd5-96c3be13f616 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:37.756323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886867ed-ddfb-44ee-8fd5-96c3be13f616 None admin] PATH_INFO: `/v3/users/a72ecaf8da434213a7bad5df8d20eedf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:37.757431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886867ed-ddfb-44ee-8fd5-96c3be13f616 None admin] RBAC: Authorizing `identity:get_user(user_id=a72ecaf8da434213a7bad5df8d20eedf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:37.757839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d096dc0-693a-4c03-b094-3ceb4eb35b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:37.758646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d096dc0-693a-4c03-b094-3ceb4eb35b4a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:37.758831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d096dc0-693a-4c03-b094-3ceb4eb35b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:37.759103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d096dc0-693a-4c03-b094-3ceb4eb35b4a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:37.759253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886867ed-ddfb-44ee-8fd5-96c3be13f616 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:37.759785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d096dc0-693a-4c03-b094-3ceb4eb35b4a None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:37.760083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d096dc0-693a-4c03-b094-3ceb4eb35b4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:37.760815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 168/328] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:37 2022] GET /identity/v3/users/a72ecaf8da434213a7bad5df8d20eedf => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:37.769387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d096dc0-693a-4c03-b094-3ceb4eb35b4a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:37.769549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886867ed-ddfb-44ee-8fd5-96c3be13f616 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:37.777372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 161/329] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:37.783012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d096dc0-693a-4c03-b094-3ceb4eb35b4a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:37.818680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:37.910629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d096dc0-693a-4c03-b094-3ceb4eb35b4a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:37.912548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886867ed-ddfb-44ee-8fd5-96c3be13f616 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:37.961068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1b80d-1ef5-491b-8383-787cf7b7a0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:37.961712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1b80d-1ef5-491b-8383-787cf7b7a0ae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:37.962019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1b80d-1ef5-491b-8383-787cf7b7a0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:37.962298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1b80d-1ef5-491b-8383-787cf7b7a0ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:37.963336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0999e-dd4b-4d12-84ff-a51ed3aa7e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:37.963505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1b80d-1ef5-491b-8383-787cf7b7a0ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:37.964463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0999e-dd4b-4d12-84ff-a51ed3aa7e53 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:37.964947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1b80d-1ef5-491b-8383-787cf7b7a0ae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:37.965069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0999e-dd4b-4d12-84ff-a51ed3aa7e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:37.965453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0999e-dd4b-4d12-84ff-a51ed3aa7e53 None admin] PATH_INFO: `/v3/system/users/a72ecaf8da434213a7bad5df8d20eedf/roles/4bbefa34d6764a6fa5b84ff6d1a4f09f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:37.966595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 169/330] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:54:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:37.967379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1b80d-1ef5-491b-8383-787cf7b7a0ae None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:37.969333 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0999e-dd4b-4d12-84ff-a51ed3aa7e53 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a72ecaf8da434213a7bad5df8d20eedf, role_id=4bbefa34d6764a6fa5b84ff6d1a4f09f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:37.972040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0999e-dd4b-4d12-84ff-a51ed3aa7e53 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:37.982998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:37.987463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 162/331] 10.222.0.16 () {58 vars in 1334 bytes} [Fri Aug 19 20:54:37 2022] PUT /identity/v3/system/users/a72ecaf8da434213a7bad5df8d20eedf/roles/4bbefa34d6764a6fa5b84ff6d1a4f09f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:37.988942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0999e-dd4b-4d12-84ff-a51ed3aa7e53 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:38.128349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1b80d-1ef5-491b-8383-787cf7b7a0ae None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:38.149795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:38.178493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe521c-a144-4fb5-9865-5f47e13296b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:38.179475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe521c-a144-4fb5-9865-5f47e13296b0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:38.179911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe521c-a144-4fb5-9865-5f47e13296b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:38.180218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0999e-dd4b-4d12-84ff-a51ed3aa7e53 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:38.180377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fe521c-a144-4fb5-9865-5f47e13296b0 None admin] PATH_INFO: `/v3/users/e7362d182871460ca31055bd28f62fcd` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:38.182306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe521c-a144-4fb5-9865-5f47e13296b0 None admin] RBAC: Authorizing `identity:get_user(user_id=e7362d182871460ca31055bd28f62fcd)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:38.184852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fe521c-a144-4fb5-9865-5f47e13296b0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:38.188595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 170/332] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:37 2022] GET /identity/v3/users/e7362d182871460ca31055bd28f62fcd => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:38.195811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe521c-a144-4fb5-9865-5f47e13296b0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:38.252662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e959d2-579a-47d2-b484-7a18e6c8a250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:38.253671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e959d2-579a-47d2-b484-7a18e6c8a250 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:38.254293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e959d2-579a-47d2-b484-7a18e6c8a250 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:38.254784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e959d2-579a-47d2-b484-7a18e6c8a250 None admin] PATH_INFO: `/v3/groups` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:38.256080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e959d2-579a-47d2-b484-7a18e6c8a250 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:38.258996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e959d2-579a-47d2-b484-7a18e6c8a250 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:38.288564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 163/333] 10.222.0.16 () {60 vars in 1134 bytes} [Fri Aug 19 20:54:37 2022] POST /identity/v3/groups => generated 226 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:54:38.294579 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:38.323882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fe521c-a144-4fb5-9865-5f47e13296b0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:38.373023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d256bc-dce4-4a08-828f-43b387211766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:38.373549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d256bc-dce4-4a08-828f-43b387211766 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:38.373866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d256bc-dce4-4a08-828f-43b387211766 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:38.374215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d256bc-dce4-4a08-828f-43b387211766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:38.375077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d256bc-dce4-4a08-828f-43b387211766 None admin] RBAC: Adding query filter params (role.id=f8c927860d6941559283c83fcaaf93d8, user.id=e7362d182871460ca31055bd28f62fcd, scope.system=all) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:38.375450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d256bc-dce4-4a08-828f-43b387211766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:38.376972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d256bc-dce4-4a08-828f-43b387211766 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:38.398955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 171/334] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Aug 19 20:54:38 2022] GET /identity/v3/role_assignments?user.id=e7362d182871460ca31055bd28f62fcd&scope.system=all&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:40.859118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade4c8-6ab2-4a13-88fa-8d03b235926d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:40.860116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade4c8-6ab2-4a13-88fa-8d03b235926d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:40.860116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ade4c8-6ab2-4a13-88fa-8d03b235926d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:40.861371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 164/335] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:40.878510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff75c-9e20-4478-89b7-6955130926c0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:40.879296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff75c-9e20-4478-89b7-6955130926c0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:40.879296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9ff75c-9e20-4478-89b7-6955130926c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:40.908905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6b9ff75c-9e20-4478-89b7-6955130926c0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:40.910184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6b9ff75c-9e20-4478-89b7-6955130926c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:41.018396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 172/336] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:41.236594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910196d6-56ec-4e32-9c84-3b015fa38b21 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:41.236829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910196d6-56ec-4e32-9c84-3b015fa38b21 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:41.237028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910196d6-56ec-4e32-9c84-3b015fa38b21 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:41.238957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 165/337] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:41.249545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca16a8-2c09-4a33-8de9-747323c6beb7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:41.250219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca16a8-2c09-4a33-8de9-747323c6beb7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:41.250688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca16a8-2c09-4a33-8de9-747323c6beb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:41.280186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f8ca16a8-2c09-4a33-8de9-747323c6beb7 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:41.282220 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f8ca16a8-2c09-4a33-8de9-747323c6beb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:41.384862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:41.396786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 173/338] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:41.469574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910196d6-56ec-4e32-9c84-3b015fa38b21 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:41.518350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41c228-f6b9-479d-b27f-9a8ce49f025b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:41.519562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41c228-f6b9-479d-b27f-9a8ce49f025b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:41.520383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41c228-f6b9-479d-b27f-9a8ce49f025b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:41.524951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 174/339] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:41.537024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bb7fd-c1dc-427a-b8fa-bb65cde29b77 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:41.538429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bb7fd-c1dc-427a-b8fa-bb65cde29b77 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:41.539250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bb7fd-c1dc-427a-b8fa-bb65cde29b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:41.579294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-641bb7fd-c1dc-427a-b8fa-bb65cde29b77 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:41.581930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-641bb7fd-c1dc-427a-b8fa-bb65cde29b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:41.744675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 175/340] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:41.753048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffc478-fdca-4fa3-ac11-99db1c706051 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:41.753929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffc478-fdca-4fa3-ac11-99db1c706051 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:41.754633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ffc478-fdca-4fa3-ac11-99db1c706051 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:41.757872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 176/341] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:41.764773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75d242-1b16-4313-a85b-1687cd49a6fe None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:41.765712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75d242-1b16-4313-a85b-1687cd49a6fe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:41.766724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75d242-1b16-4313-a85b-1687cd49a6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:41.767635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910196d6-56ec-4e32-9c84-3b015fa38b21 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:41.818984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-aa75d242-1b16-4313-a85b-1687cd49a6fe None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:41.826023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aa75d242-1b16-4313-a85b-1687cd49a6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:41.835186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849507c-1356-4335-92a4-bc7d733faf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:41.835846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849507c-1356-4335-92a4-bc7d733faf6a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:41.836144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849507c-1356-4335-92a4-bc7d733faf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:41.836407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849507c-1356-4335-92a4-bc7d733faf6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:41.865802 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849507c-1356-4335-92a4-bc7d733faf6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1849507c-1356-4335-92a4-bc7d733faf6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:41.873380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:41.875243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:41.876845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 166/342] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 407 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:41.885563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849507c-1356-4335-92a4-bc7d733faf6a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:41.896287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:42.042670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 177/343] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:42.063973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849507c-1356-4335-92a4-bc7d733faf6a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:42.114390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72081259-4a31-4fef-ba3e-03912a2c0e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:42.115031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72081259-4a31-4fef-ba3e-03912a2c0e60 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:42.115316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72081259-4a31-4fef-ba3e-03912a2c0e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:42.115588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72081259-4a31-4fef-ba3e-03912a2c0e60 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:42.116281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72081259-4a31-4fef-ba3e-03912a2c0e60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:42.116595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72081259-4a31-4fef-ba3e-03912a2c0e60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:42.118029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72081259-4a31-4fef-ba3e-03912a2c0e60 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:42.133991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75d242-1b16-4313-a85b-1687cd49a6fe None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:42.134240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 167/344] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:42.284643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:42.285868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72081259-4a31-4fef-ba3e-03912a2c0e60 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:42.397328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75d242-1b16-4313-a85b-1687cd49a6fe None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:42.426937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:42.460799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76e23-58a1-4b93-96bd-241a5a6a0ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:42.463129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76e23-58a1-4b93-96bd-241a5a6a0ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:42.463631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76e23-58a1-4b93-96bd-241a5a6a0ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:42.464137 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76e23-58a1-4b93-96bd-241a5a6a0ab9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:42.484029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72081259-4a31-4fef-ba3e-03912a2c0e60 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:42.587041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbe3f4-3097-4e64-8f72-6a469308914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:42.591078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbe3f4-3097-4e64-8f72-6a469308914f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:42.594142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbe3f4-3097-4e64-8f72-6a469308914f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:42.595067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbe3f4-3097-4e64-8f72-6a469308914f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:42.618168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb76e23-58a1-4b93-96bd-241a5a6a0ab9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:42.622353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb76e23-58a1-4b93-96bd-241a5a6a0ab9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:42.651139 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbe3f4-3097-4e64-8f72-6a469308914f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f2bbe3f4-3097-4e64-8f72-6a469308914f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:42.656955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:42.658712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:42.660305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:42.663601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 168/345] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:42.667529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbe3f4-3097-4e64-8f72-6a469308914f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:42.774301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:42.831282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 178/346] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:42.832522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76e23-58a1-4b93-96bd-241a5a6a0ab9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:42.868230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbe3f4-3097-4e64-8f72-6a469308914f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:42.923085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:42.937968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec9402-d344-4939-a11c-332883fa4ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:42.939651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec9402-d344-4939-a11c-332883fa4ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:42.940203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec9402-d344-4939-a11c-332883fa4ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:42.940854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec9402-d344-4939-a11c-332883fa4ac1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:42.956320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec9402-d344-4939-a11c-332883fa4ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5bec9402-d344-4939-a11c-332883fa4ac1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:42.963568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:42.964583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:42.965368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:42.968689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 169/347] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:42.971687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec9402-d344-4939-a11c-332883fa4ac1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:42.992181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76e23-58a1-4b93-96bd-241a5a6a0ab9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:43.041349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b736b2-17f5-40fc-aa45-b05da5917db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:43.042624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b736b2-17f5-40fc-aa45-b05da5917db5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:43.043186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b736b2-17f5-40fc-aa45-b05da5917db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:43.044089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b736b2-17f5-40fc-aa45-b05da5917db5 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:43.044969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b736b2-17f5-40fc-aa45-b05da5917db5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:43.045478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b736b2-17f5-40fc-aa45-b05da5917db5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:43.047709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b736b2-17f5-40fc-aa45-b05da5917db5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:43.057931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 179/348] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:43.059104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b736b2-17f5-40fc-aa45-b05da5917db5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:43.081756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:43.181128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec9402-d344-4939-a11c-332883fa4ac1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:43.231972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d3d7b-aca8-4155-bae5-404d65eb5445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:43.232942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3d7b-aca8-4155-bae5-404d65eb5445 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:43.234012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3d7b-aca8-4155-bae5-404d65eb5445 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:43.234493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3d7b-aca8-4155-bae5-404d65eb5445 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:43.255869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d3d7b-aca8-4155-bae5-404d65eb5445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:43.256678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b736b2-17f5-40fc-aa45-b05da5917db5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-752d3d7b-aca8-4155-bae5-404d65eb5445 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:43.265034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:43.273367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 170/349] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:43.274934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d3d7b-aca8-4155-bae5-404d65eb5445 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:43.331339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4faff7f-35fc-4265-9eef-93fd2542f685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:43.331965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faff7f-35fc-4265-9eef-93fd2542f685 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:43.332406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faff7f-35fc-4265-9eef-93fd2542f685 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:43.332863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faff7f-35fc-4265-9eef-93fd2542f685 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:43.334840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4faff7f-35fc-4265-9eef-93fd2542f685 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:43.337355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4faff7f-35fc-4265-9eef-93fd2542f685 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:43.339927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 180/350] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:43 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:43.343438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444ac8-7585-4877-8cca-3d2f4adcdc82 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:43.344192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444ac8-7585-4877-8cca-3d2f4adcdc82 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:43.344682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444ac8-7585-4877-8cca-3d2f4adcdc82 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:43.347355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 181/351] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:43.352151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52444ac8-7585-4877-8cca-3d2f4adcdc82 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:43.370401 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:43.490524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d3d7b-aca8-4155-bae5-404d65eb5445 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:43.529456 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:43.565983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52444ac8-7585-4877-8cca-3d2f4adcdc82 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:43.578274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aed01c-16e9-4c15-a0e8-97257bd31e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:43.579343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aed01c-16e9-4c15-a0e8-97257bd31e87 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:43.579817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aed01c-16e9-4c15-a0e8-97257bd31e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:43.580253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aed01c-16e9-4c15-a0e8-97257bd31e87 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:43.581474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aed01c-16e9-4c15-a0e8-97257bd31e87 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:43.582051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aed01c-16e9-4c15-a0e8-97257bd31e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:43.584327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aed01c-16e9-4c15-a0e8-97257bd31e87 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:43.599891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 171/352] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:43.601595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aed01c-16e9-4c15-a0e8-97257bd31e87 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:43.626711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8762380-39f7-4fb5-b876-1ddf9326f3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:43.627552 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8762380-39f7-4fb5-b876-1ddf9326f3fc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:43.627987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8762380-39f7-4fb5-b876-1ddf9326f3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:43.628489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8762380-39f7-4fb5-b876-1ddf9326f3fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:43.754829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8762380-39f7-4fb5-b876-1ddf9326f3fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:43.757481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8762380-39f7-4fb5-b876-1ddf9326f3fc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:43.777884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aed01c-16e9-4c15-a0e8-97257bd31e87 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:43.796404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:43.836081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0255cd-86df-432c-8a81-2f18e56853a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:43.837465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0255cd-86df-432c-8a81-2f18e56853a6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:43.837986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0255cd-86df-432c-8a81-2f18e56853a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:43.838435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0255cd-86df-432c-8a81-2f18e56853a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:43.839214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0255cd-86df-432c-8a81-2f18e56853a6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:43.839523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0255cd-86df-432c-8a81-2f18e56853a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:43.841498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0255cd-86df-432c-8a81-2f18e56853a6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:43.853590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 172/353] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:43.855124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0255cd-86df-432c-8a81-2f18e56853a6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:43.916852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:43.951810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 182/354] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:43.954649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429c7b6-accf-4e14-9f57-4a6c0a5bfea1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:43.955051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429c7b6-accf-4e14-9f57-4a6c0a5bfea1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:43.955474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0429c7b6-accf-4e14-9f57-4a6c0a5bfea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:43.984821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0429c7b6-accf-4e14-9f57-4a6c0a5bfea1 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:43.986163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0429c7b6-accf-4e14-9f57-4a6c0a5bfea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:44.032321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0255cd-86df-432c-8a81-2f18e56853a6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:44.088967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ca61-3b99-4899-b3ce-ae53298cabe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:44.090250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ca61-3b99-4899-b3ce-ae53298cabe0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:44.090250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ca61-3b99-4899-b3ce-ae53298cabe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:44.090569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ca61-3b99-4899-b3ce-ae53298cabe0 None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:44.105195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4ca61-3b99-4899-b3ce-ae53298cabe0 None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:44.108705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4ca61-3b99-4899-b3ce-ae53298cabe0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:44.110725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 183/355] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:44.112200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 173/356] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:43 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:44.113316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c93d14-abff-499b-834a-7a86189272e1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:44.113481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c93d14-abff-499b-834a-7a86189272e1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:44.113638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ca61-3b99-4899-b3ce-ae53298cabe0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:44.113934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c93d14-abff-499b-834a-7a86189272e1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:44.116852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 184/357] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:44.120493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c93d14-abff-499b-834a-7a86189272e1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:44.193370 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:44.316256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c93d14-abff-499b-834a-7a86189272e1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:44.335650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ca61-3b99-4899-b3ce-ae53298cabe0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:44.340920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:44.386983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08928a6-933a-47e8-82fc-cf04fbfe90a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:44.388135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08928a6-933a-47e8-82fc-cf04fbfe90a4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:44.388574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08928a6-933a-47e8-82fc-cf04fbfe90a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:44.389057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08928a6-933a-47e8-82fc-cf04fbfe90a4 None admin] PATH_INFO: `/v3/users/a72ecaf8da434213a7bad5df8d20eedf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:44.392561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08928a6-933a-47e8-82fc-cf04fbfe90a4 None admin] RBAC: Authorizing `identity:get_user(user_id=a72ecaf8da434213a7bad5df8d20eedf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:44.395212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08928a6-933a-47e8-82fc-cf04fbfe90a4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:44.399826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 185/358] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:54:44 2022] GET /identity/v3/users/a72ecaf8da434213a7bad5df8d20eedf => generated 293 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:44.401286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08928a6-933a-47e8-82fc-cf04fbfe90a4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:44.424870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a9bfb-00c9-40f0-b4da-73302bb059d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:44.426299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a9bfb-00c9-40f0-b4da-73302bb059d2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:44.426937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a9bfb-00c9-40f0-b4da-73302bb059d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:44.427522 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a9bfb-00c9-40f0-b4da-73302bb059d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:44.560607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a9bfb-00c9-40f0-b4da-73302bb059d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:44.563722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a9bfb-00c9-40f0-b4da-73302bb059d2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:44.581285 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08928a6-933a-47e8-82fc-cf04fbfe90a4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:44.635358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:44.642797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900f4b1-6291-46f7-84ac-c4755e482795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:44.643868 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900f4b1-6291-46f7-84ac-c4755e482795 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:44.644355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900f4b1-6291-46f7-84ac-c4755e482795 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:44.644847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900f4b1-6291-46f7-84ac-c4755e482795 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:44.646956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900f4b1-6291-46f7-84ac-c4755e482795 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:44.649555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900f4b1-6291-46f7-84ac-c4755e482795 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:44.652675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 186/359] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:44 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:44.654297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900f4b1-6291-46f7-84ac-c4755e482795 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:44.728798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:44.757830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 174/360] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:44.759536 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756002f1-ab92-4c7d-bc1a-fa03a37eba72 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:44.759955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756002f1-ab92-4c7d-bc1a-fa03a37eba72 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:44.760372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756002f1-ab92-4c7d-bc1a-fa03a37eba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:44.786639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-756002f1-ab92-4c7d-bc1a-fa03a37eba72 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:44.788327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-756002f1-ab92-4c7d-bc1a-fa03a37eba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:44.813163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900f4b1-6291-46f7-84ac-c4755e482795 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:44.869016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5542769-b193-4693-bf46-a707eb281718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:44.870223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5542769-b193-4693-bf46-a707eb281718 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:44.870710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5542769-b193-4693-bf46-a707eb281718 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:44.871204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5542769-b193-4693-bf46-a707eb281718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:44.872586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5542769-b193-4693-bf46-a707eb281718 None admin] RBAC: Adding query filter params (group.id=16472761615b490298f06d4fd9012d83, role.id=a631c57af3494b9c955799e0488518ae, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:44.873139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5542769-b193-4693-bf46-a707eb281718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:44.876505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5542769-b193-4693-bf46-a707eb281718 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:44.888281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 187/361] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Aug 19 20:54:44 2022] GET /identity/v3/role_assignments?group.id=16472761615b490298f06d4fd9012d83&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=a631c57af3494b9c955799e0488518ae => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:44.890285 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5542769-b193-4693-bf46-a707eb281718 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:44.894704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 175/362] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:44.895808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756002f1-ab92-4c7d-bc1a-fa03a37eba72 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:44.987652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:45.085445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756002f1-ab92-4c7d-bc1a-fa03a37eba72 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:45.089156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5542769-b193-4693-bf46-a707eb281718 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:45.111341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:45.139945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97aa55c-5650-46c0-abc7-e1087eec823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:45.140580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97aa55c-5650-46c0-abc7-e1087eec823e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:45.140939 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97aa55c-5650-46c0-abc7-e1087eec823e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:45.141411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97aa55c-5650-46c0-abc7-e1087eec823e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:45.142306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97aa55c-5650-46c0-abc7-e1087eec823e None admin] RBAC: Adding query filter params (role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f, user.id=a72ecaf8da434213a7bad5df8d20eedf, scope.system=all) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:45.143008 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97aa55c-5650-46c0-abc7-e1087eec823e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:45.144251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97aa55c-5650-46c0-abc7-e1087eec823e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:45.169389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 188/363] 10.222.0.16 () {56 vars in 1308 bytes} [Fri Aug 19 20:54:44 2022] GET /identity/v3/role_assignments?user.id=a72ecaf8da434213a7bad5df8d20eedf&scope.system=all&role.id=4bbefa34d6764a6fa5b84ff6d1a4f09f => generated 510 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:45.170056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97aa55c-5650-46c0-abc7-e1087eec823e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:45.180225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f788fa-2ce3-40cc-a85e-fc0dcdbe8fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:45.181198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f788fa-2ce3-40cc-a85e-fc0dcdbe8fbd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:45.181608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f788fa-2ce3-40cc-a85e-fc0dcdbe8fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:45.183509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f788fa-2ce3-40cc-a85e-fc0dcdbe8fbd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:45.213810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f788fa-2ce3-40cc-a85e-fc0dcdbe8fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f3f788fa-2ce3-40cc-a85e-fc0dcdbe8fbd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:45.222658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:45.223656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:45.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:45.231525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 176/364] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:45.235360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f788fa-2ce3-40cc-a85e-fc0dcdbe8fbd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:45.407377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97aa55c-5650-46c0-abc7-e1087eec823e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:45.442381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f788fa-2ce3-40cc-a85e-fc0dcdbe8fbd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:45.480451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:45.483000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dc8d9-6816-4ac8-a413-0a484a1f9d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:45.484449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dc8d9-6816-4ac8-a413-0a484a1f9d8b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:45.484745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dc8d9-6816-4ac8-a413-0a484a1f9d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:45.485017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018dc8d9-6816-4ac8-a413-0a484a1f9d8b None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:45.487383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018dc8d9-6816-4ac8-a413-0a484a1f9d8b None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:45.489310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018dc8d9-6816-4ac8-a413-0a484a1f9d8b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:45.491250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 189/365] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:45 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:45.492063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dc8d9-6816-4ac8-a413-0a484a1f9d8b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:45.492945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb495e8-964a-49c3-a8f5-59f8971868c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:45.493892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb495e8-964a-49c3-a8f5-59f8971868c6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:45.494172 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb495e8-964a-49c3-a8f5-59f8971868c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:45.494447 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb495e8-964a-49c3-a8f5-59f8971868c6 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:45.495644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb495e8-964a-49c3-a8f5-59f8971868c6 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:45.497127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb495e8-964a-49c3-a8f5-59f8971868c6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:45.499392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 177/366] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:45 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:45.502426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb495e8-964a-49c3-a8f5-59f8971868c6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:45.562548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:45.664933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018dc8d9-6816-4ac8-a413-0a484a1f9d8b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:45.673150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb495e8-964a-49c3-a8f5-59f8971868c6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:45.719894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732cd86-1956-4f9f-8610-6d6c6c5a4284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:45.720998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732cd86-1956-4f9f-8610-6d6c6c5a4284 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:45.721521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732cd86-1956-4f9f-8610-6d6c6c5a4284 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:45.722402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732cd86-1956-4f9f-8610-6d6c6c5a4284 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:45.742009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df816523-b5c5-42ed-bb6f-f6a4ad705a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:45.742443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732cd86-1956-4f9f-8610-6d6c6c5a4284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:45.743107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df816523-b5c5-42ed-bb6f-f6a4ad705a22 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:45.743887 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df816523-b5c5-42ed-bb6f-f6a4ad705a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:45.746074 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df816523-b5c5-42ed-bb6f-f6a4ad705a22 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:45.747086 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df816523-b5c5-42ed-bb6f-f6a4ad705a22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:45.747481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df816523-b5c5-42ed-bb6f-f6a4ad705a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:45.748841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df816523-b5c5-42ed-bb6f-f6a4ad705a22 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c732cd86-1956-4f9f-8610-6d6c6c5a4284 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:45.754563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:45.755705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:45.756648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:45.758459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 190/367] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:54:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:45.766006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 178/368] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:45.766006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df816523-b5c5-42ed-bb6f-f6a4ad705a22 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:45.766274 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732cd86-1956-4f9f-8610-6d6c6c5a4284 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:45.845096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:45.913983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:45.917518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732cd86-1956-4f9f-8610-6d6c6c5a4284 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:45.924019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df816523-b5c5-42ed-bb6f-f6a4ad705a22 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:45.960583 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba1a73-9bd4-4414-9639-ca3f6ff0a4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:45.961196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba1a73-9bd4-4414-9639-ca3f6ff0a4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:45.961475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba1a73-9bd4-4414-9639-ca3f6ff0a4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:45.961739 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba1a73-9bd4-4414-9639-ca3f6ff0a4d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:45.962496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba1a73-9bd4-4414-9639-ca3f6ff0a4d1 None admin] RBAC: Adding query filter params (group.id=16472761615b490298f06d4fd9012d83, role.id=f8c927860d6941559283c83fcaaf93d8, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:45.962798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba1a73-9bd4-4414-9639-ca3f6ff0a4d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:45.964246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba1a73-9bd4-4414-9639-ca3f6ff0a4d1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:45.972074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 191/369] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Aug 19 20:54:45 2022] GET /identity/v3/role_assignments?group.id=16472761615b490298f06d4fd9012d83&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:45.972831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba1a73-9bd4-4414-9639-ca3f6ff0a4d1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:45.979396 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551de17d-22fa-4de4-8641-d2c85b2eb5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:45.980434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551de17d-22fa-4de4-8641-d2c85b2eb5b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:45.980924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551de17d-22fa-4de4-8641-d2c85b2eb5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:45.981416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551de17d-22fa-4de4-8641-d2c85b2eb5b3 None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:45.984557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551de17d-22fa-4de4-8641-d2c85b2eb5b3 None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:45.986661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551de17d-22fa-4de4-8641-d2c85b2eb5b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:45.990226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 179/370] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:45 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:45.991354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551de17d-22fa-4de4-8641-d2c85b2eb5b3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:46.147080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551de17d-22fa-4de4-8641-d2c85b2eb5b3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:46.149369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba1a73-9bd4-4414-9639-ca3f6ff0a4d1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:46.190318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:46.208359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0fc3a-b054-4455-b41b-f0129eec0cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:46.209325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0fc3a-b054-4455-b41b-f0129eec0cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:46.209742 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0fc3a-b054-4455-b41b-f0129eec0cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:46.210144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0fc3a-b054-4455-b41b-f0129eec0cb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:46.211695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb24fa8-2673-4a70-a0ea-de778029648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:46.212330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb24fa8-2673-4a70-a0ea-de778029648d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:46.212614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb24fa8-2673-4a70-a0ea-de778029648d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:46.212887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb24fa8-2673-4a70-a0ea-de778029648d None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:46.213981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb24fa8-2673-4a70-a0ea-de778029648d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:46.214294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb24fa8-2673-4a70-a0ea-de778029648d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:46.216676 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb24fa8-2673-4a70-a0ea-de778029648d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:46.227603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 192/371] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:46.229216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb24fa8-2673-4a70-a0ea-de778029648d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:46.292267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:46.328000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0fc3a-b054-4455-b41b-f0129eec0cb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:46.329436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0fc3a-b054-4455-b41b-f0129eec0cb6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:46.442841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb24fa8-2673-4a70-a0ea-de778029648d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:46.445460 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 180/372] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:46.446386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0fc3a-b054-4455-b41b-f0129eec0cb6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:46.476905 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:46.516610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76504a2b-b724-4c3d-914a-7eb53356f43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:46.517904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76504a2b-b724-4c3d-914a-7eb53356f43c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:46.518492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76504a2b-b724-4c3d-914a-7eb53356f43c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:46.519071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76504a2b-b724-4c3d-914a-7eb53356f43c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:46.520368 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76504a2b-b724-4c3d-914a-7eb53356f43c None admin] RBAC: Adding query filter params (group.id=16472761615b490298f06d4fd9012d83, role.id=f8c927860d6941559283c83fcaaf93d8, scope.project.id=4ff250f07e37455ba8616d6434ffbef6) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:46.520928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76504a2b-b724-4c3d-914a-7eb53356f43c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:46.523120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76504a2b-b724-4c3d-914a-7eb53356f43c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:46.533606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 193/373] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Aug 19 20:54:46 2022] GET /identity/v3/role_assignments?group.id=16472761615b490298f06d4fd9012d83&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 260 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:46.535307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76504a2b-b724-4c3d-914a-7eb53356f43c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:46.623806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0fc3a-b054-4455-b41b-f0129eec0cb6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:46.700359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7d878-311e-4604-96fc-2d69fcf0bcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:46.701060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7d878-311e-4604-96fc-2d69fcf0bcae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:46.701471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7d878-311e-4604-96fc-2d69fcf0bcae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:46.701924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7d878-311e-4604-96fc-2d69fcf0bcae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:46.825010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:46.827277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7d878-311e-4604-96fc-2d69fcf0bcae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:46.829183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7d878-311e-4604-96fc-2d69fcf0bcae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:46.849399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76504a2b-b724-4c3d-914a-7eb53356f43c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:46.887141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:46.925386 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73701c1d-9b3e-4656-b59e-64b428759b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:46.926499 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73701c1d-9b3e-4656-b59e-64b428759b3e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:46.926791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73701c1d-9b3e-4656-b59e-64b428759b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:46.927071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73701c1d-9b3e-4656-b59e-64b428759b3e None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:46.931188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73701c1d-9b3e-4656-b59e-64b428759b3e None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:46.933995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73701c1d-9b3e-4656-b59e-64b428759b3e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:46.938282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 194/374] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:46 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:46.949093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73701c1d-9b3e-4656-b59e-64b428759b3e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:47.015716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 181/375] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:47.022019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7d878-311e-4604-96fc-2d69fcf0bcae None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:47.145042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7d878-311e-4604-96fc-2d69fcf0bcae None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:47.238670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:47.282657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fd36d-c1a2-4059-990f-bca95209c360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:47.283208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fd36d-c1a2-4059-990f-bca95209c360 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:47.283599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fd36d-c1a2-4059-990f-bca95209c360 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:47.283877 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785fd36d-c1a2-4059-990f-bca95209c360 None admin] PATH_INFO: `/v3/projects/4f484db3eb0540218c8a18fc8601facf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:47.285257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fd36d-c1a2-4059-990f-bca95209c360 None admin] RBAC: Authorizing `identity:get_project(project_id=4f484db3eb0540218c8a18fc8601facf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:47.286757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785fd36d-c1a2-4059-990f-bca95209c360 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:47.288775 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 182/376] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:47 2022] GET /identity/v3/projects/4f484db3eb0540218c8a18fc8601facf => generated 344 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:47.294314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fd36d-c1a2-4059-990f-bca95209c360 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:47.316221 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73701c1d-9b3e-4656-b59e-64b428759b3e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:47.403773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27356a81-36a9-4442-bf2c-5e14c6b6a42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:47.404872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356a81-36a9-4442-bf2c-5e14c6b6a42c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:47.405487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356a81-36a9-4442-bf2c-5e14c6b6a42c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:47.405976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27356a81-36a9-4442-bf2c-5e14c6b6a42c None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:47.419164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27356a81-36a9-4442-bf2c-5e14c6b6a42c None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:47.422382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27356a81-36a9-4442-bf2c-5e14c6b6a42c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:47.425556 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 195/377] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:46 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:47.431941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27356a81-36a9-4442-bf2c-5e14c6b6a42c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:47.500396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:47.505258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785fd36d-c1a2-4059-990f-bca95209c360 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:47.547375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a6440-2c87-48c8-8a3a-c72e3b002e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:47.548332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a6440-2c87-48c8-8a3a-c72e3b002e30 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:47.548792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a6440-2c87-48c8-8a3a-c72e3b002e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:47.549242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a6440-2c87-48c8-8a3a-c72e3b002e30 None admin] PATH_INFO: `/v3/groups/00b7d70a456d4a4ea5c976dfa45056ab` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:47.555244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a6440-2c87-48c8-8a3a-c72e3b002e30 None admin] RBAC: Authorizing `identity:get_group(group_id=00b7d70a456d4a4ea5c976dfa45056ab)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:47.557174 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a6440-2c87-48c8-8a3a-c72e3b002e30 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:47.559555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 183/378] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:47 2022] GET /identity/v3/groups/00b7d70a456d4a4ea5c976dfa45056ab => generated 229 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:47.561136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e011d-5293-4604-b238-63a6061c6f13 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:47.562278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e011d-5293-4604-b238-63a6061c6f13 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:47.562885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e011d-5293-4604-b238-63a6061c6f13 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:47.569035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 184/379] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:47.570374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e011d-5293-4604-b238-63a6061c6f13 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:47.637149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:47.654922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27356a81-36a9-4442-bf2c-5e14c6b6a42c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:47.693828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e011d-5293-4604-b238-63a6061c6f13 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:47.726320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d75c6-c5ad-401e-9728-c439ebaa92ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:47.727642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d75c6-c5ad-401e-9728-c439ebaa92ef None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:47.728329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d75c6-c5ad-401e-9728-c439ebaa92ef None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:47.728892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d75c6-c5ad-401e-9728-c439ebaa92ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:47.730571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d75c6-c5ad-401e-9728-c439ebaa92ef None admin] RBAC: Adding query filter params (group.id=16472761615b490298f06d4fd9012d83, role.id=a631c57af3494b9c955799e0488518ae, scope.project.id=4ff250f07e37455ba8616d6434ffbef6) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:47.731358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d75c6-c5ad-401e-9728-c439ebaa92ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:47.735170 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d75c6-c5ad-401e-9728-c439ebaa92ef None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:47.735784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954da8ef-872c-4958-a17c-c8be7f813366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:47.736608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954da8ef-872c-4958-a17c-c8be7f813366 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:47.737062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954da8ef-872c-4958-a17c-c8be7f813366 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:47.737538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954da8ef-872c-4958-a17c-c8be7f813366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:47.738501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954da8ef-872c-4958-a17c-c8be7f813366 None admin] RBAC: Adding query filter params (group.id=00b7d70a456d4a4ea5c976dfa45056ab, role.id=8b350845f331423cb0494e3449b58310, scope.project.id=4f484db3eb0540218c8a18fc8601facf) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:47.738999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954da8ef-872c-4958-a17c-c8be7f813366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:47.740667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954da8ef-872c-4958-a17c-c8be7f813366 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:47.747546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 185/381] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Aug 19 20:54:47 2022] GET /identity/v3/role_assignments?group.id=00b7d70a456d4a4ea5c976dfa45056ab&scope.project.id=4f484db3eb0540218c8a18fc8601facf&role.id=8b350845f331423cb0494e3449b58310 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:47.747737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 196/381] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Aug 19 20:54:47 2022] GET /identity/v3/role_assignments?group.id=16472761615b490298f06d4fd9012d83&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=a631c57af3494b9c955799e0488518ae => generated 260 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:47.757817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f422319-c14f-4bb2-beb5-f32ee8d6c0c5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:47.757817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f422319-c14f-4bb2-beb5-f32ee8d6c0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:47.757817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f422319-c14f-4bb2-beb5-f32ee8d6c0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:47.803751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1f422319-c14f-4bb2-beb5-f32ee8d6c0c5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:47.805844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1f422319-c14f-4bb2-beb5-f32ee8d6c0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:47.940244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 197/382] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:48.524722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954da8ef-872c-4958-a17c-c8be7f813366 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:48.681124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954da8ef-872c-4958-a17c-c8be7f813366 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:48.713173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:48.736581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50132113-755c-4ea8-8abc-904493634fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:48.737249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50132113-755c-4ea8-8abc-904493634fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:48.737530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50132113-755c-4ea8-8abc-904493634fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:48.737832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50132113-755c-4ea8-8abc-904493634fd6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:48.758729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50132113-755c-4ea8-8abc-904493634fd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-50132113-755c-4ea8-8abc-904493634fd6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:48.764280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:48.767157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:48.769130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 186/383] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:48.774343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f422319-c14f-4bb2-beb5-f32ee8d6c0c5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:48.812981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:48.913661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0dfd9-b8c8-473a-934a-25a5f5b1b320 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:48.914417 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0dfd9-b8c8-473a-934a-25a5f5b1b320 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:48.914911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0dfd9-b8c8-473a-934a-25a5f5b1b320 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:48.917783 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 187/384] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:48.926794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351731f9-01f7-4bde-9ee1-36ab13e032a1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:48.927279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351731f9-01f7-4bde-9ee1-36ab13e032a1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:48.927730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351731f9-01f7-4bde-9ee1-36ab13e032a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:48.938420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f422319-c14f-4bb2-beb5-f32ee8d6c0c5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:48.963233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-351731f9-01f7-4bde-9ee1-36ab13e032a1 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:48.966045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-351731f9-01f7-4bde-9ee1-36ab13e032a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:49.031362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44fbcc-b77a-4f81-a841-1cf181c3009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:49.032015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44fbcc-b77a-4f81-a841-1cf181c3009e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:49.032293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44fbcc-b77a-4f81-a841-1cf181c3009e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:49.032563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44fbcc-b77a-4f81-a841-1cf181c3009e None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:49.033254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44fbcc-b77a-4f81-a841-1cf181c3009e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:49.033557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44fbcc-b77a-4f81-a841-1cf181c3009e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:49.037814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44fbcc-b77a-4f81-a841-1cf181c3009e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:49.064474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 198/385] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:49.069658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44fbcc-b77a-4f81-a841-1cf181c3009e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:49.081696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 188/386] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:49.083358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814734d0-208b-40a9-bb69-2f4d478b4f2d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:49.089787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814734d0-208b-40a9-bb69-2f4d478b4f2d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:49.089787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814734d0-208b-40a9-bb69-2f4d478b4f2d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:49.094270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 189/387] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:49 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:49.099629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd17d9b-4aa7-413b-96c6-bf32e6800da8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:49.099915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd17d9b-4aa7-413b-96c6-bf32e6800da8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:49.100195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd17d9b-4aa7-413b-96c6-bf32e6800da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:49.134097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4dd17d9b-4aa7-413b-96c6-bf32e6800da8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:49.135348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4dd17d9b-4aa7-413b-96c6-bf32e6800da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:49.258096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:49.290191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 190/388] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:49.300439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44fbcc-b77a-4f81-a841-1cf181c3009e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:49.344232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0040a2f-9c49-4fe4-afce-8f9d524b0299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:49.344880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0040a2f-9c49-4fe4-afce-8f9d524b0299 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:49.345162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0040a2f-9c49-4fe4-afce-8f9d524b0299 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:49.345442 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0040a2f-9c49-4fe4-afce-8f9d524b0299 None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:49.350594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0040a2f-9c49-4fe4-afce-8f9d524b0299 None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:49.352083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0040a2f-9c49-4fe4-afce-8f9d524b0299 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:49.354100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 199/389] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:49 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:49.360044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd17d9b-4aa7-413b-96c6-bf32e6800da8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:49.422237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:49.498051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd17d9b-4aa7-413b-96c6-bf32e6800da8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:49.542513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cdfaf-1280-4d32-bd0b-75cd7e6ac344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:49.543256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cdfaf-1280-4d32-bd0b-75cd7e6ac344 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:49.543442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cdfaf-1280-4d32-bd0b-75cd7e6ac344 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:49.543724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512cdfaf-1280-4d32-bd0b-75cd7e6ac344 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:49.545089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0040a2f-9c49-4fe4-afce-8f9d524b0299 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:49.546833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512cdfaf-1280-4d32-bd0b-75cd7e6ac344 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:49.548310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512cdfaf-1280-4d32-bd0b-75cd7e6ac344 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:49.550776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 191/390] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:49 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:49.556337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cdfaf-1280-4d32-bd0b-75cd7e6ac344 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:49.695112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0040a2f-9c49-4fe4-afce-8f9d524b0299 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:49.742182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa7071-99e3-433c-8671-4be6ee0d234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:49.743193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa7071-99e3-433c-8671-4be6ee0d234f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:49.743995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa7071-99e3-433c-8671-4be6ee0d234f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:49.744423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fa7071-99e3-433c-8671-4be6ee0d234f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:49.756973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fa7071-99e3-433c-8671-4be6ee0d234f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a9fa7071-99e3-433c-8671-4be6ee0d234f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:49.763115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:49.764183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:49.765198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:49.767338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 200/391] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:54:49.767888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512cdfaf-1280-4d32-bd0b-75cd7e6ac344 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:49.768432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa7071-99e3-433c-8671-4be6ee0d234f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:49.784175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:49.789639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:49.836425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8bdf-e372-4502-9731-dfe04687bbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:49.837495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8bdf-e372-4502-9731-dfe04687bbcb None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:49.837948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8bdf-e372-4502-9731-dfe04687bbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:49.838413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8bdf-e372-4502-9731-dfe04687bbcb None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/groups/16472761615b490298f06d4fd9012d83/roles/a631c57af3494b9c955799e0488518ae` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:49.841070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae8bdf-e372-4502-9731-dfe04687bbcb None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd6e674b6d240749abdcbbd0b748d15, group_id=16472761615b490298f06d4fd9012d83, role_id=a631c57af3494b9c955799e0488518ae)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:49.843468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae8bdf-e372-4502-9731-dfe04687bbcb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:49.860918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 192/392] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Aug 19 20:54:49 2022] PUT /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/groups/16472761615b490298f06d4fd9012d83/roles/a631c57af3494b9c955799e0488518ae => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:49.862226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8bdf-e372-4502-9731-dfe04687bbcb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:49.976127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fa7071-99e3-433c-8671-4be6ee0d234f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:50.010132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8bdf-e372-4502-9731-dfe04687bbcb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:50.061310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c08743-bb35-4669-8d7e-4f7980b674b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:50.062667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c08743-bb35-4669-8d7e-4f7980b674b1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.063104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c08743-bb35-4669-8d7e-4f7980b674b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.063553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c08743-bb35-4669-8d7e-4f7980b674b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.064572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c08743-bb35-4669-8d7e-4f7980b674b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:50.065034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c08743-bb35-4669-8d7e-4f7980b674b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:50.066782 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c08743-bb35-4669-8d7e-4f7980b674b1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:50.078934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 193/393] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:50.089038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c08743-bb35-4669-8d7e-4f7980b674b1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:50.094432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c660c1-5270-40fc-bdd1-309a61691aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:50.095555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c660c1-5270-40fc-bdd1-309a61691aef None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.096043 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c660c1-5270-40fc-bdd1-309a61691aef None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.097730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c660c1-5270-40fc-bdd1-309a61691aef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.120330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c660c1-5270-40fc-bdd1-309a61691aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-51c660c1-5270-40fc-bdd1-309a61691aef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:50.128485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:50.129460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:50.130271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:50.133097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 201/394] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:50.151738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c660c1-5270-40fc-bdd1-309a61691aef None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:50.177827 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:50.267790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:50.277033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c08743-bb35-4669-8d7e-4f7980b674b1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:50.315348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c660c1-5270-40fc-bdd1-309a61691aef None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:50.326506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e85a96-d9d6-4f85-aef6-a9bed431826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:50.328404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e85a96-d9d6-4f85-aef6-a9bed431826b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.328965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e85a96-d9d6-4f85-aef6-a9bed431826b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.329447 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e85a96-d9d6-4f85-aef6-a9bed431826b None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.330959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e85a96-d9d6-4f85-aef6-a9bed431826b None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:50.332659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e85a96-d9d6-4f85-aef6-a9bed431826b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:50.335015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 194/395] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:50 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:50.347233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e85a96-d9d6-4f85-aef6-a9bed431826b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:50.366956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e53748-d2f8-407a-bc59-d4f12215823b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:50.367828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e53748-d2f8-407a-bc59-d4f12215823b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.368117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e53748-d2f8-407a-bc59-d4f12215823b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.368658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e53748-d2f8-407a-bc59-d4f12215823b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.369482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e53748-d2f8-407a-bc59-d4f12215823b None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:50.370233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e53748-d2f8-407a-bc59-d4f12215823b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:50.375022 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e53748-d2f8-407a-bc59-d4f12215823b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:50.402916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 202/396] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:50.410995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e53748-d2f8-407a-bc59-d4f12215823b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:50.543270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:50.547790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e85a96-d9d6-4f85-aef6-a9bed431826b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:50.600467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e6da8-2a48-46cd-abcc-4522fc628e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:50.601107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e6da8-2a48-46cd-abcc-4522fc628e6c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.601384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e6da8-2a48-46cd-abcc-4522fc628e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.601657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e6da8-2a48-46cd-abcc-4522fc628e6c None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.603316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e6da8-2a48-46cd-abcc-4522fc628e6c None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:50.604789 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e6da8-2a48-46cd-abcc-4522fc628e6c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:50.606641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 195/397] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:50 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:50.607886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5eea96-efbd-47d8-8f79-efe71a78c10e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.608264 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e53748-d2f8-407a-bc59-d4f12215823b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:50.608421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5eea96-efbd-47d8-8f79-efe71a78c10e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.608583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5eea96-efbd-47d8-8f79-efe71a78c10e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.616733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 196/398] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:50.616733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae73e9-7a29-4d3b-b1ca-85efeaadb7ea None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.616733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae73e9-7a29-4d3b-b1ca-85efeaadb7ea None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.616733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae73e9-7a29-4d3b-b1ca-85efeaadb7ea None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.616733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 197/399] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:50.616733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae73e9-7a29-4d3b-b1ca-85efeaadb7ea None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:50.661716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:50.669649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ec706-9cf0-4f07-8771-4b59ee396610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:50.671204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ec706-9cf0-4f07-8771-4b59ee396610 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.671607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ec706-9cf0-4f07-8771-4b59ee396610 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.672440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ec706-9cf0-4f07-8771-4b59ee396610 None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.673933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ec706-9cf0-4f07-8771-4b59ee396610 None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:50.676007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ec706-9cf0-4f07-8771-4b59ee396610 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:50.683562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 203/400] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:50 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:50.683562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25484ba4-0b78-430e-a3d1-5b8bf304838f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.683562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25484ba4-0b78-430e-a3d1-5b8bf304838f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.683562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25484ba4-0b78-430e-a3d1-5b8bf304838f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.701592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-25484ba4-0b78-430e-a3d1-5b8bf304838f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:50.703472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-25484ba4-0b78-430e-a3d1-5b8bf304838f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:50.748312 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae73e9-7a29-4d3b-b1ca-85efeaadb7ea None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:50.787005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 204/401] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:50.791594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af80d88-83e0-434e-a0ea-11487fc662f8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.791594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af80d88-83e0-434e-a0ea-11487fc662f8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.791594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af80d88-83e0-434e-a0ea-11487fc662f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.817811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b97980-58f6-4f9d-be45-9a9e9a8fae75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:50.819484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b97980-58f6-4f9d-be45-9a9e9a8fae75 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:50.820196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b97980-58f6-4f9d-be45-9a9e9a8fae75 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:50.820546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5af80d88-83e0-434e-a0ea-11487fc662f8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:50.821230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b97980-58f6-4f9d-be45-9a9e9a8fae75 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6/groups/16472761615b490298f06d4fd9012d83/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:50.822503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5af80d88-83e0-434e-a0ea-11487fc662f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:50.825345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b97980-58f6-4f9d-be45-9a9e9a8fae75 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ff250f07e37455ba8616d6434ffbef6, group_id=16472761615b490298f06d4fd9012d83, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:50.830518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b97980-58f6-4f9d-be45-9a9e9a8fae75 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:50.863804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 198/402] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Aug 19 20:54:50 2022] PUT /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6/groups/16472761615b490298f06d4fd9012d83/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:50.863804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b97980-58f6-4f9d-be45-9a9e9a8fae75 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:50.966374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:51.048643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 205/403] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:51.112360 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af80d88-83e0-434e-a0ea-11487fc662f8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:51.154482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b97980-58f6-4f9d-be45-9a9e9a8fae75 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:51.214709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ca08c-f1be-4447-aadc-c8988bc350a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:51.215498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ca08c-f1be-4447-aadc-c8988bc350a1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:51.215816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ca08c-f1be-4447-aadc-c8988bc350a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:51.216131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ca08c-f1be-4447-aadc-c8988bc350a1 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:51.218021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ca08c-f1be-4447-aadc-c8988bc350a1 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:51.219634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ca08c-f1be-4447-aadc-c8988bc350a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:51.221428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 199/404] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:50 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:51.229410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ca08c-f1be-4447-aadc-c8988bc350a1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:51.253913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:51.350262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af80d88-83e0-434e-a0ea-11487fc662f8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:51.399508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3433118-e96f-4605-8219-4084f4ac98f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:51.400291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3433118-e96f-4605-8219-4084f4ac98f9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:51.400651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3433118-e96f-4605-8219-4084f4ac98f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:51.400929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3433118-e96f-4605-8219-4084f4ac98f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:51.407167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:51.425486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3433118-e96f-4605-8219-4084f4ac98f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:51.428121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ca08c-f1be-4447-aadc-c8988bc350a1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c3433118-e96f-4605-8219-4084f4ac98f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:51.434952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:51.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:51.439079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 206/405] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:51.451145 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3433118-e96f-4605-8219-4084f4ac98f9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:51.540311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf21041-e68d-4c30-ba5b-4bb2173a07e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:51.549569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf21041-e68d-4c30-ba5b-4bb2173a07e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:51.549569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf21041-e68d-4c30-ba5b-4bb2173a07e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:51.549569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf21041-e68d-4c30-ba5b-4bb2173a07e7 None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/groups/16472761615b490298f06d4fd9012d83/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:51.552170 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf21041-e68d-4c30-ba5b-4bb2173a07e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd6e674b6d240749abdcbbd0b748d15, group_id=16472761615b490298f06d4fd9012d83, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:51.562204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf21041-e68d-4c30-ba5b-4bb2173a07e7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:51.596296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 200/406] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Aug 19 20:54:51 2022] PUT /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15/groups/16472761615b490298f06d4fd9012d83/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 20:54:51.613011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3433118-e96f-4605-8219-4084f4ac98f9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:51.642418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf21041-e68d-4c30-ba5b-4bb2173a07e7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:51.646307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:51.679737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba85f0e-219a-4ba7-b3e6-2866fec0d800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:51.680613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba85f0e-219a-4ba7-b3e6-2866fec0d800 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:51.680883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba85f0e-219a-4ba7-b3e6-2866fec0d800 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:51.682686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba85f0e-219a-4ba7-b3e6-2866fec0d800 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:51.684143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba85f0e-219a-4ba7-b3e6-2866fec0d800 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:51.684747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba85f0e-219a-4ba7-b3e6-2866fec0d800 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:51.686229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba85f0e-219a-4ba7-b3e6-2866fec0d800 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:51.700882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 207/407] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:51.703420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba85f0e-219a-4ba7-b3e6-2866fec0d800 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:51.830658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf21041-e68d-4c30-ba5b-4bb2173a07e7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:51.878089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a536faa-eb29-4c2d-ba08-3e1976fe44ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:51.878702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a536faa-eb29-4c2d-ba08-3e1976fe44ab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:51.878985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a536faa-eb29-4c2d-ba08-3e1976fe44ab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:51.879259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a536faa-eb29-4c2d-ba08-3e1976fe44ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:51.892033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a536faa-eb29-4c2d-ba08-3e1976fe44ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:51.897688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba85f0e-219a-4ba7-b3e6-2866fec0d800 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1a536faa-eb29-4c2d-ba08-3e1976fe44ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:51.898437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:51.899971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:51.901571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 201/408] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:54:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:51.905721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a536faa-eb29-4c2d-ba08-3e1976fe44ab None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:51.933322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:51.956898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1bb23-0468-4b81-bb63-da3c38df8c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:51.957553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1bb23-0468-4b81-bb63-da3c38df8c16 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:51.957862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1bb23-0468-4b81-bb63-da3c38df8c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:51.958140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1bb23-0468-4b81-bb63-da3c38df8c16 None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:51.959952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1bb23-0468-4b81-bb63-da3c38df8c16 None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:51.961408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1bb23-0468-4b81-bb63-da3c38df8c16 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:51.963044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 208/409] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:51 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:51.971164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1bb23-0468-4b81-bb63-da3c38df8c16 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:52.096881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:52.125111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a536faa-eb29-4c2d-ba08-3e1976fe44ab None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:52.197166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd4e30-6963-42db-8efd-a537623798fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:52.197697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd4e30-6963-42db-8efd-a537623798fb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:52.198000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd4e30-6963-42db-8efd-a537623798fb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:52.198295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd4e30-6963-42db-8efd-a537623798fb None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:52.198989 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd4e30-6963-42db-8efd-a537623798fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:52.199296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd4e30-6963-42db-8efd-a537623798fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:52.200688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd4e30-6963-42db-8efd-a537623798fb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:52.213570 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1bb23-0468-4b81-bb63-da3c38df8c16 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:52.225547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 202/410] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:52.240560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd4e30-6963-42db-8efd-a537623798fb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:52.267560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739bbac-11e1-4f18-a7ac-7db6fb9559f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:52.268840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739bbac-11e1-4f18-a7ac-7db6fb9559f1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:52.269346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739bbac-11e1-4f18-a7ac-7db6fb9559f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:52.269876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0739bbac-11e1-4f18-a7ac-7db6fb9559f1 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:52.272758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739bbac-11e1-4f18-a7ac-7db6fb9559f1 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:52.276083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0739bbac-11e1-4f18-a7ac-7db6fb9559f1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:52.279688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 209/411] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:51 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:52.288573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739bbac-11e1-4f18-a7ac-7db6fb9559f1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:52.417431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:52.454228 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0739bbac-11e1-4f18-a7ac-7db6fb9559f1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:52.469148 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd4e30-6963-42db-8efd-a537623798fb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:52.482609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:52.526575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd21860-3138-4fd5-a4db-d035707e3fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:52.527758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd21860-3138-4fd5-a4db-d035707e3fc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:52.528259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd21860-3138-4fd5-a4db-d035707e3fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:52.528748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd21860-3138-4fd5-a4db-d035707e3fc2 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6/groups/16472761615b490298f06d4fd9012d83/roles/a631c57af3494b9c955799e0488518ae` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:52.528965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d67ae-7308-45dc-9b1f-9c7c31010ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:52.530002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d67ae-7308-45dc-9b1f-9c7c31010ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:52.531096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d67ae-7308-45dc-9b1f-9c7c31010ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:52.531624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d67ae-7308-45dc-9b1f-9c7c31010ac1 None admin] PATH_INFO: `/v3/groups/00b7d70a456d4a4ea5c976dfa45056ab` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:52.533013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd21860-3138-4fd5-a4db-d035707e3fc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4ff250f07e37455ba8616d6434ffbef6, group_id=16472761615b490298f06d4fd9012d83, role_id=a631c57af3494b9c955799e0488518ae)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:52.533493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d67ae-7308-45dc-9b1f-9c7c31010ac1 None admin] RBAC: Authorizing `identity:get_group(group_id=00b7d70a456d4a4ea5c976dfa45056ab)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:52.535148 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d67ae-7308-45dc-9b1f-9c7c31010ac1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:52.536283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd21860-3138-4fd5-a4db-d035707e3fc2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:52.537423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 203/412] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:52 2022] GET /identity/v3/groups/00b7d70a456d4a4ea5c976dfa45056ab => generated 229 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:52.538854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981f9f8-e2d3-4a6d-a1cd-de54a6aef64b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:52.539268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981f9f8-e2d3-4a6d-a1cd-de54a6aef64b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:52.539681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981f9f8-e2d3-4a6d-a1cd-de54a6aef64b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:52.541843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 204/413] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:52.543347 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981f9f8-e2d3-4a6d-a1cd-de54a6aef64b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:52.563076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 210/414] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Aug 19 20:54:52 2022] PUT /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6/groups/16472761615b490298f06d4fd9012d83/roles/a631c57af3494b9c955799e0488518ae => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:52.572046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd88f0e-eb4c-40ff-b4d9-62e4187ec066 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:52.572846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd88f0e-eb4c-40ff-b4d9-62e4187ec066 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:52.573460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd88f0e-eb4c-40ff-b4d9-62e4187ec066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:52.635020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bcd88f0e-eb4c-40ff-b4d9-62e4187ec066 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:52.636696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bcd88f0e-eb4c-40ff-b4d9-62e4187ec066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:52.675484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981f9f8-e2d3-4a6d-a1cd-de54a6aef64b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:52.735831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcae1d0-49a3-4abd-9273-2947d2dd3478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:52.737578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcae1d0-49a3-4abd-9273-2947d2dd3478 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:52.738241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcae1d0-49a3-4abd-9273-2947d2dd3478 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:52.739873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcae1d0-49a3-4abd-9273-2947d2dd3478 None admin] PATH_INFO: `/v3/projects/4f484db3eb0540218c8a18fc8601facf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:52.742271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcae1d0-49a3-4abd-9273-2947d2dd3478 None admin] RBAC: Authorizing `identity:get_project(project_id=4f484db3eb0540218c8a18fc8601facf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:52.744417 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcae1d0-49a3-4abd-9273-2947d2dd3478 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:52.748094 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 205/415] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:52 2022] GET /identity/v3/projects/4f484db3eb0540218c8a18fc8601facf => generated 344 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:52.756073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcae1d0-49a3-4abd-9273-2947d2dd3478 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:52.791892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:52.880291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 211/416] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:52.929527 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcae1d0-49a3-4abd-9273-2947d2dd3478 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:52.978313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c9df64-ff1d-4b35-bade-24622a3688aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:52.979251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9df64-ff1d-4b35-bade-24622a3688aa None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:52.979912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9df64-ff1d-4b35-bade-24622a3688aa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:52.980340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c9df64-ff1d-4b35-bade-24622a3688aa None admin] PATH_INFO: `/v3/projects/4f484db3eb0540218c8a18fc8601facf/groups/00b7d70a456d4a4ea5c976dfa45056ab/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:52.982604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9df64-ff1d-4b35-bade-24622a3688aa None admin] RBAC: Authorizing `identity:create_grant(project_id=4f484db3eb0540218c8a18fc8601facf, group_id=00b7d70a456d4a4ea5c976dfa45056ab, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:52.985026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c9df64-ff1d-4b35-bade-24622a3688aa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:53.006059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 206/417] 10.222.0.16 () {58 vars in 1442 bytes} [Fri Aug 19 20:54:52 2022] PUT /identity/v3/projects/4f484db3eb0540218c8a18fc8601facf/groups/00b7d70a456d4a4ea5c976dfa45056ab/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:54:53.305894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd88f0e-eb4c-40ff-b4d9-62e4187ec066 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:53.545654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd88f0e-eb4c-40ff-b4d9-62e4187ec066 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:53.584397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:53.661728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655f6cb-c905-4a06-877f-14ba14dc7403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:53.663658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655f6cb-c905-4a06-877f-14ba14dc7403 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:53.664043 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655f6cb-c905-4a06-877f-14ba14dc7403 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:53.664524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5655f6cb-c905-4a06-877f-14ba14dc7403 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:53.683057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad497a0-c1ea-4650-add4-37c69f92d3f9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:53.683703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad497a0-c1ea-4650-add4-37c69f92d3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:53.683703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad497a0-c1ea-4650-add4-37c69f92d3f9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:53.685316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 207/418] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:53.694629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5655f6cb-c905-4a06-877f-14ba14dc7403 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5655f6cb-c905-4a06-877f-14ba14dc7403 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:53.699520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:53.701064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:53.703475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 212/419] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:53.707361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18877ee9-1cc6-4d63-8363-8f2113d25d73 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:53.707863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18877ee9-1cc6-4d63-8363-8f2113d25d73 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:53.708344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18877ee9-1cc6-4d63-8363-8f2113d25d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:53.713607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655f6cb-c905-4a06-877f-14ba14dc7403 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:53.745830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-18877ee9-1cc6-4d63-8363-8f2113d25d73 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:53.747998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-18877ee9-1cc6-4d63-8363-8f2113d25d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:53.901957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 208/420] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:53.950274 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5655f6cb-c905-4a06-877f-14ba14dc7403 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:54.015478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebab578-a25f-485d-abe3-5832bf7155e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:54.016293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebab578-a25f-485d-abe3-5832bf7155e5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:54.016572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebab578-a25f-485d-abe3-5832bf7155e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:54.016848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebab578-a25f-485d-abe3-5832bf7155e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:54.017542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebab578-a25f-485d-abe3-5832bf7155e5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:54.017877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebab578-a25f-485d-abe3-5832bf7155e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:54.019301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebab578-a25f-485d-abe3-5832bf7155e5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:54.038031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 213/421] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:54.046549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18877ee9-1cc6-4d63-8363-8f2113d25d73 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:54.050302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:54.191414 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18877ee9-1cc6-4d63-8363-8f2113d25d73 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:54.239731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1acd409-2bb7-4cbf-b456-7fc2212bf656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:54.240412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acd409-2bb7-4cbf-b456-7fc2212bf656 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:54.240694 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acd409-2bb7-4cbf-b456-7fc2212bf656 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:54.240967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1acd409-2bb7-4cbf-b456-7fc2212bf656 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:54.336005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acd409-2bb7-4cbf-b456-7fc2212bf656 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:54.337571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1acd409-2bb7-4cbf-b456-7fc2212bf656 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:54.339832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:54.391917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebab578-a25f-485d-abe3-5832bf7155e5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:54.462467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:54.476475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 209/422] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:54.477834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf414c-5b6c-4c66-b480-8579dcb81e45 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:54.478130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf414c-5b6c-4c66-b480-8579dcb81e45 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:54.478404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf414c-5b6c-4c66-b480-8579dcb81e45 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:54.480035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 210/423] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:54.490506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d1296-214c-4320-9204-8442321242a0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:54.491346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d1296-214c-4320-9204-8442321242a0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:54.492000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d1296-214c-4320-9204-8442321242a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:54.526594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ad0d1296-214c-4320-9204-8442321242a0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:54.528750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ad0d1296-214c-4320-9204-8442321242a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:54.536342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebab578-a25f-485d-abe3-5832bf7155e5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:54.586120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2828c0-ba32-4424-ad7b-73aff5f3d910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:54.586883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2828c0-ba32-4424-ad7b-73aff5f3d910 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:54.587188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2828c0-ba32-4424-ad7b-73aff5f3d910 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:54.587661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2828c0-ba32-4424-ad7b-73aff5f3d910 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:54.603020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2828c0-ba32-4424-ad7b-73aff5f3d910 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bd2828c0-ba32-4424-ad7b-73aff5f3d910 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:54.611702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:54.612668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:54.613480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:54.619390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 214/424] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:54:54.619390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2828c0-ba32-4424-ad7b-73aff5f3d910 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:54.634063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 211/425] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:54.635565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d1296-214c-4320-9204-8442321242a0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:54.690722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:54.841341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:54.850280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d1296-214c-4320-9204-8442321242a0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:54.885475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2828c0-ba32-4424-ad7b-73aff5f3d910 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:54.900819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfc754-30fd-41f1-b27e-06cd3821ccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:54.901885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfc754-30fd-41f1-b27e-06cd3821ccc1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:54.902308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfc754-30fd-41f1-b27e-06cd3821ccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:54.902721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfc754-30fd-41f1-b27e-06cd3821ccc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:54.903546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfc754-30fd-41f1-b27e-06cd3821ccc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:54.905356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfc754-30fd-41f1-b27e-06cd3821ccc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:54.906974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cfc754-30fd-41f1-b27e-06cd3821ccc1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:54.939292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 212/426] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:54.955246 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfc754-30fd-41f1-b27e-06cd3821ccc1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:54.958349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4781bae-85ec-4674-8093-cc0da1c8542a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:54.959752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4781bae-85ec-4674-8093-cc0da1c8542a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:54.960483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4781bae-85ec-4674-8093-cc0da1c8542a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:54.961175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4781bae-85ec-4674-8093-cc0da1c8542a None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:54.969270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4781bae-85ec-4674-8093-cc0da1c8542a None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:54.976822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4781bae-85ec-4674-8093-cc0da1c8542a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:54.986534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 215/427] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:54 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:54.996348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4781bae-85ec-4674-8093-cc0da1c8542a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:55.137001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:55.145290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfc754-30fd-41f1-b27e-06cd3821ccc1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:55.175519 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4781bae-85ec-4674-8093-cc0da1c8542a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:55.185335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41341c8-0728-4afc-82a1-61d6ee7536ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:55.187084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41341c8-0728-4afc-82a1-61d6ee7536ad None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:55.187482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41341c8-0728-4afc-82a1-61d6ee7536ad None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:55.187943 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41341c8-0728-4afc-82a1-61d6ee7536ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:55.217023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a8e7e-f32c-4c3c-966d-84a2884602a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:55.218846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a8e7e-f32c-4c3c-966d-84a2884602a6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:55.219875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a8e7e-f32c-4c3c-966d-84a2884602a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:55.220156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54a8e7e-f32c-4c3c-966d-84a2884602a6 None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:55.221499 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a8e7e-f32c-4c3c-966d-84a2884602a6 None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:55.223123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54a8e7e-f32c-4c3c-966d-84a2884602a6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:55.225280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 216/428] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:54 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:55.226359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a8e7e-f32c-4c3c-966d-84a2884602a6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:55.268135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:55.276594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41341c8-0728-4afc-82a1-61d6ee7536ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:55.278159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41341c8-0728-4afc-82a1-61d6ee7536ad None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:55.366320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54a8e7e-f32c-4c3c-966d-84a2884602a6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:55.411347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714a677-88a6-40c7-8041-daf1a0aa1695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:55.413035 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714a677-88a6-40c7-8041-daf1a0aa1695 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:55.413889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714a677-88a6-40c7-8041-daf1a0aa1695 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:55.414987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714a677-88a6-40c7-8041-daf1a0aa1695 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:55.436092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714a677-88a6-40c7-8041-daf1a0aa1695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:55.437409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b714a677-88a6-40c7-8041-daf1a0aa1695 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:55.445000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:55.445957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 19 20:54:55.446763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:55.453249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 217/429] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:55.453249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714a677-88a6-40c7-8041-daf1a0aa1695 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:55.457499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 213/430] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:55.459320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41341c8-0728-4afc-82a1-61d6ee7536ad None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:55.581181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:55.598905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714a677-88a6-40c7-8041-daf1a0aa1695 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:55.605190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41341c8-0728-4afc-82a1-61d6ee7536ad None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:55.637093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baf42e-9282-48c2-a2db-0d5e821a6066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:55.637703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baf42e-9282-48c2-a2db-0d5e821a6066 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:55.638034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baf42e-9282-48c2-a2db-0d5e821a6066 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:55.638309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82baf42e-9282-48c2-a2db-0d5e821a6066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:55.639050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baf42e-9282-48c2-a2db-0d5e821a6066 None admin] RBAC: Adding query filter params (group.id=16472761615b490298f06d4fd9012d83, role.id=a631c57af3494b9c955799e0488518ae, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:55.639901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baf42e-9282-48c2-a2db-0d5e821a6066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:55.641372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82baf42e-9282-48c2-a2db-0d5e821a6066 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:55.648159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 218/431] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Aug 19 20:54:55 2022] GET /identity/v3/role_assignments?group.id=16472761615b490298f06d4fd9012d83&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=a631c57af3494b9c955799e0488518ae => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:55.649004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baf42e-9282-48c2-a2db-0d5e821a6066 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:55.651214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33812c12-2ded-4acb-8954-0299dac0183b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:55.652266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33812c12-2ded-4acb-8954-0299dac0183b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:55.652808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33812c12-2ded-4acb-8954-0299dac0183b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:55.653281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33812c12-2ded-4acb-8954-0299dac0183b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:55.654234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33812c12-2ded-4acb-8954-0299dac0183b None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:55.654803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33812c12-2ded-4acb-8954-0299dac0183b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:55.657033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33812c12-2ded-4acb-8954-0299dac0183b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:55.672693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 214/432] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:54:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:55.674893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33812c12-2ded-4acb-8954-0299dac0183b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:55.744955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:55.763657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82baf42e-9282-48c2-a2db-0d5e821a6066 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:55.802957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33812c12-2ded-4acb-8954-0299dac0183b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:55.805875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cc12b-c38e-460f-b014-845b418aca77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:55.807248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cc12b-c38e-460f-b014-845b418aca77 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:55.807248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cc12b-c38e-460f-b014-845b418aca77 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:55.807248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cc12b-c38e-460f-b014-845b418aca77 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:55.808608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cc12b-c38e-460f-b014-845b418aca77 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:55.810199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cc12b-c38e-460f-b014-845b418aca77 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:55.817624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 219/433] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:55 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:55.817624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21104b63-f4ef-471d-9ee8-acdf4a4f399e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:55.817624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21104b63-f4ef-471d-9ee8-acdf4a4f399e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:55.817624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21104b63-f4ef-471d-9ee8-acdf4a4f399e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:55.818714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 220/434] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:55.823367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21104b63-f4ef-471d-9ee8-acdf4a4f399e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:55.848551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:55.855578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5a612-a745-4060-831f-607d45b1771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:55.856812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5a612-a745-4060-831f-607d45b1771f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:55.857358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5a612-a745-4060-831f-607d45b1771f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:55.857837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5a612-a745-4060-831f-607d45b1771f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:55.938620 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5a612-a745-4060-831f-607d45b1771f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:55.940872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5a612-a745-4060-831f-607d45b1771f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:55.955646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21104b63-f4ef-471d-9ee8-acdf4a4f399e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:55.996254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50659de8-deb5-400e-8343-a7c63fcfda2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:55.997046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659de8-deb5-400e-8343-a7c63fcfda2b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:55.997330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659de8-deb5-400e-8343-a7c63fcfda2b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:55.997600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50659de8-deb5-400e-8343-a7c63fcfda2b None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:55.999634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50659de8-deb5-400e-8343-a7c63fcfda2b None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:56.001303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50659de8-deb5-400e-8343-a7c63fcfda2b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:56.008502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 221/435] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:55 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:56.009434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:56.011249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dd60b-605a-4ebe-878f-0efb8c967ecc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:56.011537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dd60b-605a-4ebe-878f-0efb8c967ecc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:56.011867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6dd60b-605a-4ebe-878f-0efb8c967ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:56.038563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2b6dd60b-605a-4ebe-878f-0efb8c967ecc None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:56.040163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2b6dd60b-605a-4ebe-878f-0efb8c967ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:56.080921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 215/436] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 20:54:56.086949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5a612-a745-4060-831f-607d45b1771f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:56.132536 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 222/437] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:54:56.133235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dd60b-605a-4ebe-878f-0efb8c967ecc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:56.157388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:56.231219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5a612-a745-4060-831f-607d45b1771f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:56.265977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6dd60b-605a-4ebe-878f-0efb8c967ecc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:56.284331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc4c3f-c522-41ec-a0fe-eb76ffc6dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:56.284953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc4c3f-c522-41ec-a0fe-eb76ffc6dfbd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:56.285286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc4c3f-c522-41ec-a0fe-eb76ffc6dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:56.285507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc4c3f-c522-41ec-a0fe-eb76ffc6dfbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:56.286300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc4c3f-c522-41ec-a0fe-eb76ffc6dfbd None admin] RBAC: Adding query filter params (group.id=16472761615b490298f06d4fd9012d83, role.id=f8c927860d6941559283c83fcaaf93d8, scope.project.id=4ff250f07e37455ba8616d6434ffbef6) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:56.286613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc4c3f-c522-41ec-a0fe-eb76ffc6dfbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:56.288081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc4c3f-c522-41ec-a0fe-eb76ffc6dfbd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:56.302116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 216/438] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Aug 19 20:54:56 2022] GET /identity/v3/role_assignments?group.id=16472761615b490298f06d4fd9012d83&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 611 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:56.302116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21aba11-823f-4915-aadb-cfc09ee9990d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:56.302116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21aba11-823f-4915-aadb-cfc09ee9990d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:56.302116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21aba11-823f-4915-aadb-cfc09ee9990d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:56.302116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 217/439] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:54:56.305826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3641-b2d0-4e8b-8878-d2de7dbb1d0a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:56.305826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3641-b2d0-4e8b-8878-d2de7dbb1d0a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:56.305826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b3641-b2d0-4e8b-8878-d2de7dbb1d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:56.328113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0a7b3641-b2d0-4e8b-8878-d2de7dbb1d0a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:54:56.329384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0a7b3641-b2d0-4e8b-8878-d2de7dbb1d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:54:56.331317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea8a3f-71f1-47a8-b17e-0e010688220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:56.332683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea8a3f-71f1-47a8-b17e-0e010688220a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:56.333182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea8a3f-71f1-47a8-b17e-0e010688220a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:56.333684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddea8a3f-71f1-47a8-b17e-0e010688220a None admin] PATH_INFO: `/v3/projects/9cd6e674b6d240749abdcbbd0b748d15` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:56.335808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea8a3f-71f1-47a8-b17e-0e010688220a None admin] RBAC: Authorizing `identity:get_project(project_id=9cd6e674b6d240749abdcbbd0b748d15)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:56.338616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddea8a3f-71f1-47a8-b17e-0e010688220a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:56.347340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 223/440] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:56 2022] GET /identity/v3/projects/9cd6e674b6d240749abdcbbd0b748d15 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:56.349708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea8a3f-71f1-47a8-b17e-0e010688220a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:56.354739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:56.431483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 218/441] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:54:56.432592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b3641-b2d0-4e8b-8878-d2de7dbb1d0a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:56.494297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:56.516402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddea8a3f-71f1-47a8-b17e-0e010688220a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:56.574861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b19378-6439-4ec6-bad2-6f8ef25bd8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:56.575505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b19378-6439-4ec6-bad2-6f8ef25bd8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:56.575783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b19378-6439-4ec6-bad2-6f8ef25bd8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:56.576058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b19378-6439-4ec6-bad2-6f8ef25bd8e0 None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:56.578718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b19378-6439-4ec6-bad2-6f8ef25bd8e0 None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:56.580039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b19378-6439-4ec6-bad2-6f8ef25bd8e0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:56.582845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 224/442] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:56 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:56.590605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b19378-6439-4ec6-bad2-6f8ef25bd8e0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:56.594676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b3641-b2d0-4e8b-8878-d2de7dbb1d0a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:56.650544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f71342-5c97-41e4-bf20-7c59e25e778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:56.651481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f71342-5c97-41e4-bf20-7c59e25e778f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:56.651882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f71342-5c97-41e4-bf20-7c59e25e778f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:56.652285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f71342-5c97-41e4-bf20-7c59e25e778f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:56.665754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f71342-5c97-41e4-bf20-7c59e25e778f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f4f71342-5c97-41e4-bf20-7c59e25e778f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:56.671616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:56.672499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:54:56.673274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:54:56.674626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 219/443] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:54:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:56.680067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f71342-5c97-41e4-bf20-7c59e25e778f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:56.694945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:56.734078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b19378-6439-4ec6-bad2-6f8ef25bd8e0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:56.790006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02d1e8-e7c8-4d86-b318-9c1b91da31d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:56.790986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d1e8-e7c8-4d86-b318-9c1b91da31d2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:56.791471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d1e8-e7c8-4d86-b318-9c1b91da31d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:56.791954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02d1e8-e7c8-4d86-b318-9c1b91da31d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:56.793336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02d1e8-e7c8-4d86-b318-9c1b91da31d2 None admin] RBAC: Adding query filter params (group.id=16472761615b490298f06d4fd9012d83, role.id=f8c927860d6941559283c83fcaaf93d8, scope.project.id=9cd6e674b6d240749abdcbbd0b748d15) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:56.793912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02d1e8-e7c8-4d86-b318-9c1b91da31d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:56.796631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02d1e8-e7c8-4d86-b318-9c1b91da31d2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:56.797183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f71342-5c97-41e4-bf20-7c59e25e778f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:56.806712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 225/444] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Aug 19 20:54:56 2022] GET /identity/v3/role_assignments?group.id=16472761615b490298f06d4fd9012d83&scope.project.id=9cd6e674b6d240749abdcbbd0b748d15&role.id=f8c927860d6941559283c83fcaaf93d8 => generated 611 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:56.808031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02d1e8-e7c8-4d86-b318-9c1b91da31d2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:56.834504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:56.861497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ecafa-2098-4de3-be5e-a112c8d2e406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:56.862315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ecafa-2098-4de3-be5e-a112c8d2e406 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:56.862599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ecafa-2098-4de3-be5e-a112c8d2e406 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:56.862941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861ecafa-2098-4de3-be5e-a112c8d2e406 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:56.864345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ecafa-2098-4de3-be5e-a112c8d2e406 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:56.864345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ecafa-2098-4de3-be5e-a112c8d2e406 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:56.865445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861ecafa-2098-4de3-be5e-a112c8d2e406 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:56.882747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 220/445] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:54:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:56.884192 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ecafa-2098-4de3-be5e-a112c8d2e406 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:56.968850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02d1e8-e7c8-4d86-b318-9c1b91da31d2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:57.002530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:57.018073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15d091-a335-498f-ac1b-dd3f9bab1c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:57.018733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d091-a335-498f-ac1b-dd3f9bab1c25 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:57.019008 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d091-a335-498f-ac1b-dd3f9bab1c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:57.019550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15d091-a335-498f-ac1b-dd3f9bab1c25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:57.032763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15d091-a335-498f-ac1b-dd3f9bab1c25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3c15d091-a335-498f-ac1b-dd3f9bab1c25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:54:57.039051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:54:57.041482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:54:57.043073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 226/446] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:54:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:54:57.049843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15d091-a335-498f-ac1b-dd3f9bab1c25 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:57.053318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861ecafa-2098-4de3-be5e-a112c8d2e406 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:57.113376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63512bcc-5530-48bd-9880-b19ef08ab025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:57.115238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63512bcc-5530-48bd-9880-b19ef08ab025 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:57.115721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63512bcc-5530-48bd-9880-b19ef08ab025 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:57.116419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63512bcc-5530-48bd-9880-b19ef08ab025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:57.178433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:57.218405 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15d091-a335-498f-ac1b-dd3f9bab1c25 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:57.229400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63512bcc-5530-48bd-9880-b19ef08ab025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:57.233028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63512bcc-5530-48bd-9880-b19ef08ab025 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:57.274242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625ed25-fab7-47f2-8736-9919b7e03189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:57.274835 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625ed25-fab7-47f2-8736-9919b7e03189 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:57.275103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625ed25-fab7-47f2-8736-9919b7e03189 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:57.275375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625ed25-fab7-47f2-8736-9919b7e03189 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:57.276042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625ed25-fab7-47f2-8736-9919b7e03189 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:57.276345 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625ed25-fab7-47f2-8736-9919b7e03189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:57.277741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625ed25-fab7-47f2-8736-9919b7e03189 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:57.288857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 227/447] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:54:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:57.295136 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625ed25-fab7-47f2-8736-9919b7e03189 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:57.351220 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:57.402054 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 221/448] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 19 20:54:57.408381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63512bcc-5530-48bd-9880-b19ef08ab025 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:57.458705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625ed25-fab7-47f2-8736-9919b7e03189 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:57.512528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64369a06-8c58-4add-b805-2a5b78d25ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:57.513146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64369a06-8c58-4add-b805-2a5b78d25ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:57.513632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64369a06-8c58-4add-b805-2a5b78d25ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:57.514894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64369a06-8c58-4add-b805-2a5b78d25ac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:57.527535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:57.553478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63512bcc-5530-48bd-9880-b19ef08ab025 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:57.606140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a38442-ac4c-4a34-bd6c-190f88b3d238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:57.606784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a38442-ac4c-4a34-bd6c-190f88b3d238 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:57.607066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a38442-ac4c-4a34-bd6c-190f88b3d238 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:57.607336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a38442-ac4c-4a34-bd6c-190f88b3d238 None admin] PATH_INFO: `/v3/projects/4ff250f07e37455ba8616d6434ffbef6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:57.610451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a38442-ac4c-4a34-bd6c-190f88b3d238 None admin] RBAC: Authorizing `identity:get_project(project_id=4ff250f07e37455ba8616d6434ffbef6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:57.612022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a38442-ac4c-4a34-bd6c-190f88b3d238 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:57.613981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 222/449] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:57 2022] GET /identity/v3/projects/4ff250f07e37455ba8616d6434ffbef6 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:57.620389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a38442-ac4c-4a34-bd6c-190f88b3d238 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:57.628714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64369a06-8c58-4add-b805-2a5b78d25ac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:57.629892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64369a06-8c58-4add-b805-2a5b78d25ac5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:57.667653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:57.738624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a38442-ac4c-4a34-bd6c-190f88b3d238 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:57.746863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 228/450] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:54:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:54:57.752973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64369a06-8c58-4add-b805-2a5b78d25ac5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:57.778435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef63795-264a-4186-af31-3488d49b6c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:57.779078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef63795-264a-4186-af31-3488d49b6c3b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:57.779354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef63795-264a-4186-af31-3488d49b6c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:57.779630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef63795-264a-4186-af31-3488d49b6c3b None admin] PATH_INFO: `/v3/groups/16472761615b490298f06d4fd9012d83` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:57.782103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef63795-264a-4186-af31-3488d49b6c3b None admin] RBAC: Authorizing `identity:get_group(group_id=16472761615b490298f06d4fd9012d83)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:57.783612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef63795-264a-4186-af31-3488d49b6c3b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:57.785248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 223/451] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:57 2022] GET /identity/v3/groups/16472761615b490298f06d4fd9012d83 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:57.793054 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef63795-264a-4186-af31-3488d49b6c3b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:57.798862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:57.866503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64369a06-8c58-4add-b805-2a5b78d25ac5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:57.902667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188bae7-36ca-4bf8-9b64-9f75bfe29694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:57.903301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188bae7-36ca-4bf8-9b64-9f75bfe29694 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:57.903578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188bae7-36ca-4bf8-9b64-9f75bfe29694 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:57.903846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188bae7-36ca-4bf8-9b64-9f75bfe29694 None admin] PATH_INFO: `/v3/projects/4f484db3eb0540218c8a18fc8601facf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:57.905899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188bae7-36ca-4bf8-9b64-9f75bfe29694 None admin] RBAC: Authorizing `identity:get_project(project_id=4f484db3eb0540218c8a18fc8601facf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:57.907369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188bae7-36ca-4bf8-9b64-9f75bfe29694 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:57.909052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 229/452] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:54:57 2022] GET /identity/v3/projects/4f484db3eb0540218c8a18fc8601facf => generated 344 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:57.915493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188bae7-36ca-4bf8-9b64-9f75bfe29694 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:57.945287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef63795-264a-4186-af31-3488d49b6c3b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:57.959271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:57.992327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de5869-b507-4e80-a8ba-d0cccd04df32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:57.992937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de5869-b507-4e80-a8ba-d0cccd04df32 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:57.993212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de5869-b507-4e80-a8ba-d0cccd04df32 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:57.993736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de5869-b507-4e80-a8ba-d0cccd04df32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:57.994502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de5869-b507-4e80-a8ba-d0cccd04df32 None admin] RBAC: Adding query filter params (group.id=16472761615b490298f06d4fd9012d83, role.id=a631c57af3494b9c955799e0488518ae, scope.project.id=4ff250f07e37455ba8616d6434ffbef6) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:57.994803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de5869-b507-4e80-a8ba-d0cccd04df32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:57.996257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de5869-b507-4e80-a8ba-d0cccd04df32 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:58.002983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 224/453] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Aug 19 20:54:57 2022] GET /identity/v3/role_assignments?group.id=16472761615b490298f06d4fd9012d83&scope.project.id=4ff250f07e37455ba8616d6434ffbef6&role.id=a631c57af3494b9c955799e0488518ae => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:54:58.059118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188bae7-36ca-4bf8-9b64-9f75bfe29694 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:58.068628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:54:58.109726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee600052-e791-4ae0-8b93-091685953367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:58.110440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee600052-e791-4ae0-8b93-091685953367 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:58.110717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee600052-e791-4ae0-8b93-091685953367 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:58.110992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee600052-e791-4ae0-8b93-091685953367 None admin] PATH_INFO: `/v3/groups/00b7d70a456d4a4ea5c976dfa45056ab` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:58.112307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee600052-e791-4ae0-8b93-091685953367 None admin] RBAC: Authorizing `identity:get_group(group_id=00b7d70a456d4a4ea5c976dfa45056ab)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:58.114068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee600052-e791-4ae0-8b93-091685953367 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:58.115874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 230/454] 10.222.0.16 () {56 vars in 1180 bytes} [Fri Aug 19 20:54:57 2022] GET /identity/v3/groups/00b7d70a456d4a4ea5c976dfa45056ab => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:54:58.123313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de5869-b507-4e80-a8ba-d0cccd04df32 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:54:58.272631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de5869-b507-4e80-a8ba-d0cccd04df32 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:54:58.315924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff267e-aeca-42d4-a851-323558ddc110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:54:58.316564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff267e-aeca-42d4-a851-323558ddc110 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:54:58.316845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff267e-aeca-42d4-a851-323558ddc110 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:54:58.317124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff267e-aeca-42d4-a851-323558ddc110 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:54:58.317911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff267e-aeca-42d4-a851-323558ddc110 None admin] RBAC: Adding query filter params (group.id=00b7d70a456d4a4ea5c976dfa45056ab, role.id=8b350845f331423cb0494e3449b58310, scope.project.id=4f484db3eb0540218c8a18fc8601facf) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:54:58.318234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff267e-aeca-42d4-a851-323558ddc110 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:54:58.319714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff267e-aeca-42d4-a851-323558ddc110 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:54:58.327007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 225/455] 10.222.0.16 () {56 vars in 1376 bytes} [Fri Aug 19 20:54:58 2022] GET /identity/v3/role_assignments?group.id=00b7d70a456d4a4ea5c976dfa45056ab&scope.project.id=4f484db3eb0540218c8a18fc8601facf&role.id=8b350845f331423cb0494e3449b58310 => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:00.710667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd786005-dd1b-40e8-badb-7c487419b333 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:00.711679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd786005-dd1b-40e8-badb-7c487419b333 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:00.712157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd786005-dd1b-40e8-badb-7c487419b333 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:00.714049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 231/456] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:00.721516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c98ca0-ffaa-444b-bfc9-f59524d8e0ac None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:00.722457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c98ca0-ffaa-444b-bfc9-f59524d8e0ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:00.722927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c98ca0-ffaa-444b-bfc9-f59524d8e0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:00.751297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b1c98ca0-ffaa-444b-bfc9-f59524d8e0ac None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:00.752935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b1c98ca0-ffaa-444b-bfc9-f59524d8e0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:00.796280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:00.848625 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 226/457] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:55:01.249349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd786005-dd1b-40e8-badb-7c487419b333 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:01.399465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd786005-dd1b-40e8-badb-7c487419b333 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:01.483384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb822916-5174-46a9-8fc9-0cd765460ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:01.484415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb822916-5174-46a9-8fc9-0cd765460ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:01.484896 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb822916-5174-46a9-8fc9-0cd765460ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:01.485375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb822916-5174-46a9-8fc9-0cd765460ff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:01.506315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb822916-5174-46a9-8fc9-0cd765460ff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:01.508918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb822916-5174-46a9-8fc9-0cd765460ff8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-cb822916-5174-46a9-8fc9-0cd765460ff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:01.551004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:01.552736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:01.554369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:01.554369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:01.554369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:01.554369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 232/458] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:01.562944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c98ca0-ffaa-444b-bfc9-f59524d8e0ac None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:01.624486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f14134-de75-4b8b-a90e-d8f568aa8bdc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:01.625363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f14134-de75-4b8b-a90e-d8f568aa8bdc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:01.626103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f14134-de75-4b8b-a90e-d8f568aa8bdc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:01.629237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 233/459] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:01.640896 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abddb49-7e1c-4c26-a3e2-e6f646e0534f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:01.641738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abddb49-7e1c-4c26-a3e2-e6f646e0534f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:01.642816 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abddb49-7e1c-4c26-a3e2-e6f646e0534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:01.701333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6abddb49-7e1c-4c26-a3e2-e6f646e0534f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:01.704592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6abddb49-7e1c-4c26-a3e2-e6f646e0534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:01.746539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:01.792616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c98ca0-ffaa-444b-bfc9-f59524d8e0ac None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:01.835869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca0d5d-ceb4-45a7-8aa0-8821b72571ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:01.836876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca0d5d-ceb4-45a7-8aa0-8821b72571ad None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:01.837339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca0d5d-ceb4-45a7-8aa0-8821b72571ad None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:01.837885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca0d5d-ceb4-45a7-8aa0-8821b72571ad None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:01.838751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca0d5d-ceb4-45a7-8aa0-8821b72571ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:01.839245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca0d5d-ceb4-45a7-8aa0-8821b72571ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:01.840980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca0d5d-ceb4-45a7-8aa0-8821b72571ad None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:01.855377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 227/460] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:01.857076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a7ede-7d91-498f-8ef6-df09bc2dac1a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:01.858373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a7ede-7d91-498f-8ef6-df09bc2dac1a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:01.858940 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78a7ede-7d91-498f-8ef6-df09bc2dac1a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:01.860954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 228/461] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:01.862000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a7ede-7d91-498f-8ef6-df09bc2dac1a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:01.870363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 234/462] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:01.871866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75149e18-c5b8-4526-820a-e9673cc9837e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:01.872153 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75149e18-c5b8-4526-820a-e9673cc9837e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:01.872432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75149e18-c5b8-4526-820a-e9673cc9837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:01.897650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:01.922688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-75149e18-c5b8-4526-820a-e9673cc9837e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:01.924036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-75149e18-c5b8-4526-820a-e9673cc9837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:01.992045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78a7ede-7d91-498f-8ef6-df09bc2dac1a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:02.034334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1475e8-8562-41f3-99f3-bf45056622da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:02.035320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1475e8-8562-41f3-99f3-bf45056622da None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:02.035779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1475e8-8562-41f3-99f3-bf45056622da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:02.036297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1475e8-8562-41f3-99f3-bf45056622da None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:02.037200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1475e8-8562-41f3-99f3-bf45056622da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:02.039068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1475e8-8562-41f3-99f3-bf45056622da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:02.073370 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 229/463] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:55:01 2022] POST /identity/v3/users => generated 246 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:02.080296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 235/464] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:02.250109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1475e8-8562-41f3-99f3-bf45056622da None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:02.348763 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:02.384759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1475e8-8562-41f3-99f3-bf45056622da None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:02.451176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9037b-2ca7-42ad-b0a2-3023e283a7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:02.453371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9037b-2ca7-42ad-b0a2-3023e283a7af None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:02.454201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9037b-2ca7-42ad-b0a2-3023e283a7af None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:02.455007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c9037b-2ca7-42ad-b0a2-3023e283a7af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:02.477498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9037b-2ca7-42ad-b0a2-3023e283a7af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:02.480610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c9037b-2ca7-42ad-b0a2-3023e283a7af None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-58c9037b-2ca7-42ad-b0a2-3023e283a7af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:02.510253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:02.511895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:02.513548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:02.513548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:02.513548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:02.521899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 230/465] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:02.527886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75149e18-c5b8-4526-820a-e9673cc9837e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:02.640542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9037b-2ca7-42ad-b0a2-3023e283a7af None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:02.705323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:02.729319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75149e18-c5b8-4526-820a-e9673cc9837e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:02.774398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55258ee1-b55a-42b8-a62c-e51cdccfcde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:02.774999 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55258ee1-b55a-42b8-a62c-e51cdccfcde1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:02.775214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55258ee1-b55a-42b8-a62c-e51cdccfcde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:02.775469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55258ee1-b55a-42b8-a62c-e51cdccfcde1 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:02.776121 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55258ee1-b55a-42b8-a62c-e51cdccfcde1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:02.776425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55258ee1-b55a-42b8-a62c-e51cdccfcde1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:02.777838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55258ee1-b55a-42b8-a62c-e51cdccfcde1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:02.801789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 236/466] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:02.801789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588d3c0-1261-45b9-aced-ea09429fd4b9 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:02.801789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588d3c0-1261-45b9-aced-ea09429fd4b9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:02.801789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2588d3c0-1261-45b9-aced-ea09429fd4b9 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:02.805543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 237/467] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:02.807633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fa27b-5232-49ea-8828-e04fb2ca14a9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:02.807767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fa27b-5232-49ea-8828-e04fb2ca14a9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:02.808045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57fa27b-5232-49ea-8828-e04fb2ca14a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:02.825808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c9037b-2ca7-42ad-b0a2-3023e283a7af None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:02.828889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f57fa27b-5232-49ea-8828-e04fb2ca14a9 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:02.830606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f57fa27b-5232-49ea-8828-e04fb2ca14a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:02.864506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3c84d-2f9f-47cb-aadd-25d9f70d23f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:02.865588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3c84d-2f9f-47cb-aadd-25d9f70d23f3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:02.866133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3c84d-2f9f-47cb-aadd-25d9f70d23f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:02.866675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3c84d-2f9f-47cb-aadd-25d9f70d23f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:02.869069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:02.880163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3c84d-2f9f-47cb-aadd-25d9f70d23f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:02.881917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc3c84d-2f9f-47cb-aadd-25d9f70d23f3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7fc3c84d-2f9f-47cb-aadd-25d9f70d23f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:02.897699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:02.898769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:02.899593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:02.899593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:02.899593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:02.901611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 231/468] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:02.902819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3c84d-2f9f-47cb-aadd-25d9f70d23f3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:02.918005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 238/469] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:02.919130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fa27b-5232-49ea-8828-e04fb2ca14a9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:03.012295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3c84d-2f9f-47cb-aadd-25d9f70d23f3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:03.049632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:03.076871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57fa27b-5232-49ea-8828-e04fb2ca14a9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:03.081418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0d80f-e890-4264-9b2a-cf228458db3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:03.082604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0d80f-e890-4264-9b2a-cf228458db3a None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:03.082891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0d80f-e890-4264-9b2a-cf228458db3a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:03.083165 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0d80f-e890-4264-9b2a-cf228458db3a None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:03.083828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0d80f-e890-4264-9b2a-cf228458db3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:03.085303 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0d80f-e890-4264-9b2a-cf228458db3a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:03.118799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea91ba-d074-4905-857f-230fa5120996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:03.119768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea91ba-d074-4905-857f-230fa5120996 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:03.120251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea91ba-d074-4905-857f-230fa5120996 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:03.120783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea91ba-d074-4905-857f-230fa5120996 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:03.121679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea91ba-d074-4905-857f-230fa5120996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:03.122321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea91ba-d074-4905-857f-230fa5120996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:03.124578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea91ba-d074-4905-857f-230fa5120996 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:03.127188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 232/470] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:55:02 2022] POST /identity/v3/users => generated 248 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:03.139971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 239/471] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:03.147934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0d80f-e890-4264-9b2a-cf228458db3a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:03.220177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:03.329429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea91ba-d074-4905-857f-230fa5120996 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:03.379404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0d80f-e890-4264-9b2a-cf228458db3a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:03.458128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543dd14-5c92-4544-a99b-b0655b637111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:03.459509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543dd14-5c92-4544-a99b-b0655b637111 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:03.460003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543dd14-5c92-4544-a99b-b0655b637111 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:03.460471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7543dd14-5c92-4544-a99b-b0655b637111 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:03.461584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543dd14-5c92-4544-a99b-b0655b637111 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:03.462157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea91ba-d074-4905-857f-230fa5120996 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:03.464471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7543dd14-5c92-4544-a99b-b0655b637111 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:03.514785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 233/472] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:55:03 2022] POST /identity/v3/users => generated 248 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:03.516743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811a3e6-b430-42de-836f-7a8791d92a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:03.517802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811a3e6-b430-42de-836f-7a8791d92a6d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:03.518330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811a3e6-b430-42de-836f-7a8791d92a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:03.518884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811a3e6-b430-42de-836f-7a8791d92a6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:03.523559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:03.537428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811a3e6-b430-42de-836f-7a8791d92a6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:03.539408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811a3e6-b430-42de-836f-7a8791d92a6d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f811a3e6-b430-42de-836f-7a8791d92a6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:03.562504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:03.563494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:03.564336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:03.564336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:03.564336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:03.569924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 240/473] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:03.573373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543dd14-5c92-4544-a99b-b0655b637111 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:03.692480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:03.698435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7543dd14-5c92-4544-a99b-b0655b637111 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:03.753054 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcd661-e7e8-487b-a7d6-482f9ee5726c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:03.753690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcd661-e7e8-487b-a7d6-482f9ee5726c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:03.754072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcd661-e7e8-487b-a7d6-482f9ee5726c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:03.754346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcd661-e7e8-487b-a7d6-482f9ee5726c None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:03.755016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcd661-e7e8-487b-a7d6-482f9ee5726c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:03.755317 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcd661-e7e8-487b-a7d6-482f9ee5726c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:03.756723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcd661-e7e8-487b-a7d6-482f9ee5726c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:03.770917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 234/474] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:03.776509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811a3e6-b430-42de-836f-7a8791d92a6d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:03.800542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1baa0-74ab-40c1-b117-afa083fcc343 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:03.800864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1baa0-74ab-40c1-b117-afa083fcc343 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:03.801099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1baa0-74ab-40c1-b117-afa083fcc343 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:03.803356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 235/475] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:03.808722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859ab2d-3a1e-48a7-94bd-2b58f36c3d56 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:03.809011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859ab2d-3a1e-48a7-94bd-2b58f36c3d56 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:03.809266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859ab2d-3a1e-48a7-94bd-2b58f36c3d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:03.834131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0859ab2d-3a1e-48a7-94bd-2b58f36c3d56 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:03.835343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0859ab2d-3a1e-48a7-94bd-2b58f36c3d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:03.917744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 236/476] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:03.940334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811a3e6-b430-42de-836f-7a8791d92a6d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:04.011073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e26ff5-0bdd-4dbd-9e86-48923a421a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:04.011704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26ff5-0bdd-4dbd-9e86-48923a421a39 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:04.011980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26ff5-0bdd-4dbd-9e86-48923a421a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:04.012249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26ff5-0bdd-4dbd-9e86-48923a421a39 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:04.013202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e26ff5-0bdd-4dbd-9e86-48923a421a39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:04.016060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e26ff5-0bdd-4dbd-9e86-48923a421a39 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:04.056049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 241/477] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:55:03 2022] POST /identity/v3/users => generated 249 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 20:55:04.319235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859ab2d-3a1e-48a7-94bd-2b58f36c3d56 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:04.437339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:04.454572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859ab2d-3a1e-48a7-94bd-2b58f36c3d56 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:04.517893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9bc77-866d-416d-955b-b7254e1213cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:04.519371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9bc77-866d-416d-955b-b7254e1213cc None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:04.520171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9bc77-866d-416d-955b-b7254e1213cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:04.520944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa9bc77-866d-416d-955b-b7254e1213cc None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:04.522584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9bc77-866d-416d-955b-b7254e1213cc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:04.525171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa9bc77-866d-416d-955b-b7254e1213cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:04.540495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.sql.core [None req-3fa9bc77-866d-416d-955b-b7254e1213cc None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 19 20:55:04.540495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 19 20:55:04.540495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [parameters: {'id': '124cfcf26b5f4922be368ca966c70ab7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 19 20:55:04.540495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81570) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3fa9bc77-866d-416d-955b-b7254e1213cc None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Aug 19 20:55:04.572924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return ref.to_dict() Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 20:55:04.574699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application yield resource Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application next(self.gen) Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.rollback() Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.force_reraise() Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise self.value Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application session.commit() Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 19 20:55:04.576156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._prepare_impl() Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.session.flush() Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._flush(objects) Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application compat.raise_( Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application flush_context.execute() Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rec.execute(self) Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 19 20:55:04.579666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 20:55:04.583302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application conn.query(q) Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application result.read() Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 20:55:04.587780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [parameters: {'id': '124cfcf26b5f4922be368ca966c70ab7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:04.593407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 19 20:55:04.597204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:04.600206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 237/478] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Aug 19 20:55:04 2022] POST /identity/v3/roles => generated 144 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:04.600420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e26ff5-0bdd-4dbd-9e86-48923a421a39 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:04.663811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:04.764918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e26ff5-0bdd-4dbd-9e86-48923a421a39 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:04.816815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115506f6-4297-4965-adf7-598fc36aef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:04.817591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115506f6-4297-4965-adf7-598fc36aef4f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:04.817831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115506f6-4297-4965-adf7-598fc36aef4f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:04.818037 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115506f6-4297-4965-adf7-598fc36aef4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:04.838941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115506f6-4297-4965-adf7-598fc36aef4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-115506f6-4297-4965-adf7-598fc36aef4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:04.844777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:04.846440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:04.847598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 242/479] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:55:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:04.855043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9bc77-866d-416d-955b-b7254e1213cc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:05.047031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa9bc77-866d-416d-955b-b7254e1213cc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:05.055373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7132b6-388b-4f9b-a21e-919f3869448d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:05.055768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7132b6-388b-4f9b-a21e-919f3869448d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:05.055967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7132b6-388b-4f9b-a21e-919f3869448d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:05.057636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 243/480] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:05.063085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4abde-6a98-47eb-8f4d-551b9d9f94e2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:05.063406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4abde-6a98-47eb-8f4d-551b9d9f94e2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:05.063631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4abde-6a98-47eb-8f4d-551b9d9f94e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:05.099475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe820317-73f4-4144-8d0f-298407d1fd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:05.100175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe820317-73f4-4144-8d0f-298407d1fd2a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:05.100573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe820317-73f4-4144-8d0f-298407d1fd2a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:05.100953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe820317-73f4-4144-8d0f-298407d1fd2a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:05.101776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe820317-73f4-4144-8d0f-298407d1fd2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:05.102186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe820317-73f4-4144-8d0f-298407d1fd2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:05.103691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe820317-73f4-4144-8d0f-298407d1fd2a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:05.115698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9df4abde-6a98-47eb-8f4d-551b9d9f94e2 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:05.119079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9df4abde-6a98-47eb-8f4d-551b9d9f94e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:05.121422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 238/481] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:05.244994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 244/482] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:05.586849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe820317-73f4-4144-8d0f-298407d1fd2a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:05.610426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:05.792547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe820317-73f4-4144-8d0f-298407d1fd2a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:05.869084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab6d2c-dca3-414b-b133-a614d103cfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:05.870109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab6d2c-dca3-414b-b133-a614d103cfce None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:05.870596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab6d2c-dca3-414b-b133-a614d103cfce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:05.871086 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab6d2c-dca3-414b-b133-a614d103cfce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:05.895571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab6d2c-dca3-414b-b133-a614d103cfce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ecab6d2c-dca3-414b-b133-a614d103cfce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:05.906584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:05.907566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:05.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:05.909440 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 239/483] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:05.918526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4abde-6a98-47eb-8f4d-551b9d9f94e2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:05.988313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:06.104672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0f4f2-5171-4510-b997-0e6d4f0a0782 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:06.104672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0f4f2-5171-4510-b997-0e6d4f0a0782 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:06.104672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0f4f2-5171-4510-b997-0e6d4f0a0782 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:06.105645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 240/484] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:55:06.112242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00199c-b785-4e1a-a856-2b8be57ff332 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:06.112799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00199c-b785-4e1a-a856-2b8be57ff332 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:06.113404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00199c-b785-4e1a-a856-2b8be57ff332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:06.121065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4abde-6a98-47eb-8f4d-551b9d9f94e2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:06.134357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-df00199c-b785-4e1a-a856-2b8be57ff332 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:06.135985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-df00199c-b785-4e1a-a856-2b8be57ff332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:06.188949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710722c5-d158-4eb6-bf97-410179e6ca61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:06.189697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710722c5-d158-4eb6-bf97-410179e6ca61 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:06.190016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710722c5-d158-4eb6-bf97-410179e6ca61 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:06.190507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710722c5-d158-4eb6-bf97-410179e6ca61 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:06.191741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710722c5-d158-4eb6-bf97-410179e6ca61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:06.192278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710722c5-d158-4eb6-bf97-410179e6ca61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:06.194895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710722c5-d158-4eb6-bf97-410179e6ca61 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:06.212677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 245/485] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:06.229820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710722c5-d158-4eb6-bf97-410179e6ca61 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:06.260951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 241/486] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:55:06.383010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:06.393507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710722c5-d158-4eb6-bf97-410179e6ca61 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:06.463223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bba2e-05e8-4eea-bdc1-47ee358f28a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:06.464300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bba2e-05e8-4eea-bdc1-47ee358f28a3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:06.464796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bba2e-05e8-4eea-bdc1-47ee358f28a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:06.465278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bba2e-05e8-4eea-bdc1-47ee358f28a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:06.485639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bba2e-05e8-4eea-bdc1-47ee358f28a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:06.488491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bba2e-05e8-4eea-bdc1-47ee358f28a3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-566bba2e-05e8-4eea-bdc1-47ee358f28a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:06.517985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:06.519190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:06.520031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:06.520031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:06.520031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:06.520625 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 246/487] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:06.529194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00199c-b785-4e1a-a856-2b8be57ff332 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:06.534951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:06.671321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df00199c-b785-4e1a-a856-2b8be57ff332 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:06.712221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63cb2c-568d-4e47-aaa2-30d20e6b8a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:06.713224 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63cb2c-568d-4e47-aaa2-30d20e6b8a1a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:06.713837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63cb2c-568d-4e47-aaa2-30d20e6b8a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:06.714353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63cb2c-568d-4e47-aaa2-30d20e6b8a1a None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:06.715218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63cb2c-568d-4e47-aaa2-30d20e6b8a1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:06.715727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63cb2c-568d-4e47-aaa2-30d20e6b8a1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:06.717423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd63cb2c-568d-4e47-aaa2-30d20e6b8a1a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:06.730926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 242/488] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:06.737826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bba2e-05e8-4eea-bdc1-47ee358f28a3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:06.810580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668706a0-3bf7-469a-bb5f-b32cea30843b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:06.812059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668706a0-3bf7-469a-bb5f-b32cea30843b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:06.812845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668706a0-3bf7-469a-bb5f-b32cea30843b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:06.816216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 243/489] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:55:06.824665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668706a0-3bf7-469a-bb5f-b32cea30843b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:06.896118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bba2e-05e8-4eea-bdc1-47ee358f28a3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:06.914640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:06.953713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ced15-cde7-42dd-bce1-75d57d533c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:06.954522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ced15-cde7-42dd-bce1-75d57d533c82 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:06.954522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ced15-cde7-42dd-bce1-75d57d533c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:06.954816 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ced15-cde7-42dd-bce1-75d57d533c82 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:06.962808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ced15-cde7-42dd-bce1-75d57d533c82 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:06.964264 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ced15-cde7-42dd-bce1-75d57d533c82 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:06.965987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668706a0-3bf7-469a-bb5f-b32cea30843b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5d5ced15-cde7-42dd-bce1-75d57d533c82 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:06.978713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:06.979768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:06.980688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:06.980688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:06.980688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:06.980688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:06.980688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:06.980688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:06.980688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:06.980688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:06.980688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:06.980688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:06.987622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 247/490] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Aug 19 20:55:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:06.989458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c82f6-696e-48ac-b4ea-d673beb9e9d8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:06.990246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c82f6-696e-48ac-b4ea-d673beb9e9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:06.990729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c82f6-696e-48ac-b4ea-d673beb9e9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:07.017520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671eead-b782-43bf-80a0-6204a60770ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:07.018264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671eead-b782-43bf-80a0-6204a60770ce None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:07.018613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671eead-b782-43bf-80a0-6204a60770ce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:07.018843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671eead-b782-43bf-80a0-6204a60770ce None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:07.023985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1e6c82f6-696e-48ac-b4ea-d673beb9e9d8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:07.026567 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1e6c82f6-696e-48ac-b4ea-d673beb9e9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:07.031473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:07.039862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671eead-b782-43bf-80a0-6204a60770ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f671eead-b782-43bf-80a0-6204a60770ce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:07.046493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:07.048334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 244/491] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:07.049939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0754c95-fa54-485a-8fcb-ff3bbc743c49 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:07.051337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0754c95-fa54-485a-8fcb-ff3bbc743c49 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:07.051337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0754c95-fa54-485a-8fcb-ff3bbc743c49 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:07.059299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 245/492] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:07.059299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0754c95-fa54-485a-8fcb-ff3bbc743c49 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:07.139413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 248/493] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:07.140745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c82f6-696e-48ac-b4ea-d673beb9e9d8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:07.269966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c82f6-696e-48ac-b4ea-d673beb9e9d8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:07.279143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0754c95-fa54-485a-8fcb-ff3bbc743c49 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:07.316194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:07.331097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58264d-8e57-49c9-83f9-7098baf13596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:07.332213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58264d-8e57-49c9-83f9-7098baf13596 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:07.332791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58264d-8e57-49c9-83f9-7098baf13596 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:07.333220 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a411f45-7490-4943-80fd-400979d73b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:07.333443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58264d-8e57-49c9-83f9-7098baf13596 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:07.334333 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a411f45-7490-4943-80fd-400979d73b42 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:07.334779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a411f45-7490-4943-80fd-400979d73b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:07.334889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58264d-8e57-49c9-83f9-7098baf13596 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:07.335340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a411f45-7490-4943-80fd-400979d73b42 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:07.335523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58264d-8e57-49c9-83f9-7098baf13596 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:07.336563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a411f45-7490-4943-80fd-400979d73b42 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:07.337032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a411f45-7490-4943-80fd-400979d73b42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:07.338153 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58264d-8e57-49c9-83f9-7098baf13596 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:07.339128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a411f45-7490-4943-80fd-400979d73b42 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:07.349078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:07.362308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 249/494] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:07.364949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9e9f5-9f51-443f-a4e2-fe58a4a21f06 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:07.365687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9e9f5-9f51-443f-a4e2-fe58a4a21f06 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:07.366122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb9e9f5-9f51-443f-a4e2-fe58a4a21f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:07.392046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 246/495] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Aug 19 20:55:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:07.393244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a411f45-7490-4943-80fd-400979d73b42 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:07.406903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8fb9e9f5-9f51-443f-a4e2-fe58a4a21f06 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:07.409256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8fb9e9f5-9f51-443f-a4e2-fe58a4a21f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:07.506155 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 250/496] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:07.507088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9e9f5-9f51-443f-a4e2-fe58a4a21f06 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:07.571988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a411f45-7490-4943-80fd-400979d73b42 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:07.621601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fadca4-a7f2-48a7-8647-ce0963c0590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:07.622282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fadca4-a7f2-48a7-8647-ce0963c0590d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:07.622567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fadca4-a7f2-48a7-8647-ce0963c0590d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:07.622844 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fadca4-a7f2-48a7-8647-ce0963c0590d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:07.636697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fadca4-a7f2-48a7-8647-ce0963c0590d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:07.638510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fadca4-a7f2-48a7-8647-ce0963c0590d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-49fadca4-a7f2-48a7-8647-ce0963c0590d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:07.659367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:07.660306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:07.661142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:07.661142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:07.661142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:07.665443 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 247/497] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:07.666628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fadca4-a7f2-48a7-8647-ce0963c0590d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:07.670021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb9e9f5-9f51-443f-a4e2-fe58a4a21f06 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:07.709185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:07.710579 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:07.719183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae509cc-cfc6-4a2b-a165-4e4ab00c4806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:07.719446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae509cc-cfc6-4a2b-a165-4e4ab00c4806 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:07.719731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae509cc-cfc6-4a2b-a165-4e4ab00c4806 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:07.720005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae509cc-cfc6-4a2b-a165-4e4ab00c4806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:07.789564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fadca4-a7f2-48a7-8647-ce0963c0590d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:07.797103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae509cc-cfc6-4a2b-a165-4e4ab00c4806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:07.798880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae509cc-cfc6-4a2b-a165-4e4ab00c4806 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:07.831069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be61033-1e67-42f8-b7cf-a541617150bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:07.831630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be61033-1e67-42f8-b7cf-a541617150bb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:07.831910 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be61033-1e67-42f8-b7cf-a541617150bb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:07.832184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be61033-1e67-42f8-b7cf-a541617150bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:07.844650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be61033-1e67-42f8-b7cf-a541617150bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9be61033-1e67-42f8-b7cf-a541617150bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:07.849978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:07.850979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:07.851772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 248/498] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:07.852410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be61033-1e67-42f8-b7cf-a541617150bb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:07.932029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 251/499] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:07.932029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae509cc-cfc6-4a2b-a165-4e4ab00c4806 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:07.983836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:07.992982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:08.003605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be61033-1e67-42f8-b7cf-a541617150bb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:08.059340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990da785-74a4-4f51-9f01-1c2b9d691230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:08.059962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da785-74a4-4f51-9f01-1c2b9d691230 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.060241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da785-74a4-4f51-9f01-1c2b9d691230 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.060515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990da785-74a4-4f51-9f01-1c2b9d691230 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.061511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990da785-74a4-4f51-9f01-1c2b9d691230 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:08.061852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990da785-74a4-4f51-9f01-1c2b9d691230 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:08.063273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990da785-74a4-4f51-9f01-1c2b9d691230 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:08.082894 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 249/500] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:08.084511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990da785-74a4-4f51-9f01-1c2b9d691230 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:08.102391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae509cc-cfc6-4a2b-a165-4e4ab00c4806 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:08.144979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff49bd3-32a6-4f12-9735-39fe1bdfda8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:08.145744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff49bd3-32a6-4f12-9735-39fe1bdfda8b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.146699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff49bd3-32a6-4f12-9735-39fe1bdfda8b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.146699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff49bd3-32a6-4f12-9735-39fe1bdfda8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.160505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff49bd3-32a6-4f12-9735-39fe1bdfda8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fff49bd3-32a6-4f12-9735-39fe1bdfda8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:08.165380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:08.166950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 252/501] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:08.168554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff49bd3-32a6-4f12-9735-39fe1bdfda8b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:08.265101 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990da785-74a4-4f51-9f01-1c2b9d691230 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:08.277575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:08.290921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff49bd3-32a6-4f12-9735-39fe1bdfda8b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:08.301228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c2238-fbfe-46f3-bb16-ac29a569bc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:08.302439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c2238-fbfe-46f3-bb16-ac29a569bc55 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.302439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c2238-fbfe-46f3-bb16-ac29a569bc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.302439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c2238-fbfe-46f3-bb16-ac29a569bc55 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.303453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c2238-fbfe-46f3-bb16-ac29a569bc55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:08.303453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c2238-fbfe-46f3-bb16-ac29a569bc55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:08.304816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64c2238-fbfe-46f3-bb16-ac29a569bc55 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:08.314042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 250/502] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:08.314890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c2238-fbfe-46f3-bb16-ac29a569bc55 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:08.343012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:08.351221 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e3804-c96a-4f4b-a323-2ee5e3658de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:08.351867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e3804-c96a-4f4b-a323-2ee5e3658de6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.352142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e3804-c96a-4f4b-a323-2ee5e3658de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.352413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e3804-c96a-4f4b-a323-2ee5e3658de6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.364576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e3804-c96a-4f4b-a323-2ee5e3658de6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:08.366578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4e3804-c96a-4f4b-a323-2ee5e3658de6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8d4e3804-c96a-4f4b-a323-2ee5e3658de6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:08.384206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:08.385421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:08.386299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:08.386299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:08.386299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:08.386299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 253/503] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:08.386605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e3804-c96a-4f4b-a323-2ee5e3658de6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:08.441436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c2238-fbfe-46f3-bb16-ac29a569bc55 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:08.484460 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b02c332-7c7e-413f-8868-183e894bed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:08.484460 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02c332-7c7e-413f-8868-183e894bed27 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.484460 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02c332-7c7e-413f-8868-183e894bed27 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.484460 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b02c332-7c7e-413f-8868-183e894bed27 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.494010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02c332-7c7e-413f-8868-183e894bed27 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:08.495476 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b02c332-7c7e-413f-8868-183e894bed27 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:08.501560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e3804-c96a-4f4b-a323-2ee5e3658de6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7b02c332-7c7e-413f-8868-183e894bed27 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:08.509238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:08.510410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 251/504] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99676c6-82de-41b2-b533-13330f06a3c6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.516980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99676c6-82de-41b2-b533-13330f06a3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.517825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99676c6-82de-41b2-b533-13330f06a3c6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.517825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 252/505] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:08.517825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99676c6-82de-41b2-b533-13330f06a3c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:08.544830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cad223-a1e2-42f1-8826-d748abe3c764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:08.546482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cad223-a1e2-42f1-8826-d748abe3c764 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.547095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cad223-a1e2-42f1-8826-d748abe3c764 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.547460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cad223-a1e2-42f1-8826-d748abe3c764 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.548501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cad223-a1e2-42f1-8826-d748abe3c764 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:08.548953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cad223-a1e2-42f1-8826-d748abe3c764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:08.550653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cad223-a1e2-42f1-8826-d748abe3c764 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:08.563154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 254/506] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:08.564683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cad223-a1e2-42f1-8826-d748abe3c764 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:08.570746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:08.631889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:08.668542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99676c6-82de-41b2-b533-13330f06a3c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:08.696317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cad223-a1e2-42f1-8826-d748abe3c764 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:08.712331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7174f00-8e8a-4eac-9b97-efe39944131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:08.712980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7174f00-8e8a-4eac-9b97-efe39944131e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.713255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7174f00-8e8a-4eac-9b97-efe39944131e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.713554 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7174f00-8e8a-4eac-9b97-efe39944131e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.726444 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7174f00-8e8a-4eac-9b97-efe39944131e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:08.727963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7174f00-8e8a-4eac-9b97-efe39944131e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c7174f00-8e8a-4eac-9b97-efe39944131e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:08.744455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:08.745576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:08.752047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:08.752047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:08.752047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:08.752047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 253/507] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:08.752047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7174f00-8e8a-4eac-9b97-efe39944131e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:08.752386 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ec649-fe14-47ab-83ac-cd43bd76f5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:08.753422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ec649-fe14-47ab-83ac-cd43bd76f5d9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.753955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ec649-fe14-47ab-83ac-cd43bd76f5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.754460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ec649-fe14-47ab-83ac-cd43bd76f5d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.755682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ec649-fe14-47ab-83ac-cd43bd76f5d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:08.756237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ec649-fe14-47ab-83ac-cd43bd76f5d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:08.758878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ec649-fe14-47ab-83ac-cd43bd76f5d9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:08.788290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 255/508] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:08.788290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f7c01-8dbb-4dc6-b5ac-ff7c07c5d10f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.788290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f7c01-8dbb-4dc6-b5ac-ff7c07c5d10f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.788290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f7c01-8dbb-4dc6-b5ac-ff7c07c5d10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.807694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ca1f7c01-8dbb-4dc6-b5ac-ff7c07c5d10f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:08.808923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ca1f7c01-8dbb-4dc6-b5ac-ff7c07c5d10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:08.872611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7174f00-8e8a-4eac-9b97-efe39944131e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:08.887996 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:08.899797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 256/509] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:08.900905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f7c01-8dbb-4dc6-b5ac-ff7c07c5d10f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:08.921813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:08.931471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02728309-d442-436e-8f04-aaf4be8453b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:08.932735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02728309-d442-436e-8f04-aaf4be8453b1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:08.933464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02728309-d442-436e-8f04-aaf4be8453b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:08.934134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02728309-d442-436e-8f04-aaf4be8453b1 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:08.934885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02728309-d442-436e-8f04-aaf4be8453b1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:08.935196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02728309-d442-436e-8f04-aaf4be8453b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:08.937663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02728309-d442-436e-8f04-aaf4be8453b1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:08.966159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 254/510] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Aug 19 20:55:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:08.968622 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02728309-d442-436e-8f04-aaf4be8453b1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:09.090623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f7c01-8dbb-4dc6-b5ac-ff7c07c5d10f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:09.150835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02728309-d442-436e-8f04-aaf4be8453b1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:09.174606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeac1e9-5ab2-4be7-bdd0-a46cffb47094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:09.175700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeac1e9-5ab2-4be7-bdd0-a46cffb47094 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:09.176518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeac1e9-5ab2-4be7-bdd0-a46cffb47094 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:09.177807 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeac1e9-5ab2-4be7-bdd0-a46cffb47094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:09.193847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeac1e9-5ab2-4be7-bdd0-a46cffb47094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:09.195356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeac1e9-5ab2-4be7-bdd0-a46cffb47094 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:09.199655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8037824-21cd-499e-a4d6-3393cd9a00ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:09.200572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8037824-21cd-499e-a4d6-3393cd9a00ce None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:09.200910 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8037824-21cd-499e-a4d6-3393cd9a00ce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:09.201253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8037824-21cd-499e-a4d6-3393cd9a00ce None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:09.202516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8037824-21cd-499e-a4d6-3393cd9a00ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:09.203046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8037824-21cd-499e-a4d6-3393cd9a00ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:09.205103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8037824-21cd-499e-a4d6-3393cd9a00ce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fdeac1e9-5ab2-4be7-bdd0-a46cffb47094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:09.213033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:09.214778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:09.216392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:09.216392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:09.216392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:09.216392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 257/511] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:09.216392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeac1e9-5ab2-4be7-bdd0-a46cffb47094 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:09.223232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 255/512] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:09.224478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8037824-21cd-499e-a4d6-3393cd9a00ce None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:09.291187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:09.302080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:09.371420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeac1e9-5ab2-4be7-bdd0-a46cffb47094 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:09.377183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8037824-21cd-499e-a4d6-3393cd9a00ce None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:09.414555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91abc6a-1fe2-40b0-af1a-ad0217dc2dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:09.415111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91abc6a-1fe2-40b0-af1a-ad0217dc2dbf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:09.415389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91abc6a-1fe2-40b0-af1a-ad0217dc2dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:09.415660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91abc6a-1fe2-40b0-af1a-ad0217dc2dbf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:09.422932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91abc6a-1fe2-40b0-af1a-ad0217dc2dbf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:09.424418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91abc6a-1fe2-40b0-af1a-ad0217dc2dbf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:09.424521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83458e-b019-4481-a9e4-9f0ba3597aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:09.425397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83458e-b019-4481-a9e4-9f0ba3597aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:09.425691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83458e-b019-4481-a9e4-9f0ba3597aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:09.429895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83458e-b019-4481-a9e4-9f0ba3597aa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e91abc6a-1fe2-40b0-af1a-ad0217dc2dbf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:09.439101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:09.440179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:09.441094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:09.441094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:09.441094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:09.441094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:09.441094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:09.441094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:09.441094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:09.441094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:09.441094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 258/513] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:09.441094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91abc6a-1fe2-40b0-af1a-ad0217dc2dbf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:09.530758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83458e-b019-4481-a9e4-9f0ba3597aa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:09.532511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83458e-b019-4481-a9e4-9f0ba3597aa2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:09.559550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91abc6a-1fe2-40b0-af1a-ad0217dc2dbf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:09.569933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:09.589308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:09.601623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d8929-82ea-49d9-8c24-47d5b3dd71b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:09.602286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d8929-82ea-49d9-8c24-47d5b3dd71b1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:09.602561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d8929-82ea-49d9-8c24-47d5b3dd71b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:09.603154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d8929-82ea-49d9-8c24-47d5b3dd71b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:09.616011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d8929-82ea-49d9-8c24-47d5b3dd71b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:09.617522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d8929-82ea-49d9-8c24-47d5b3dd71b1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-591d8929-82ea-49d9-8c24-47d5b3dd71b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:09.638390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:09.639507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:09.642060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:09.642060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:09.642060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:09.642060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 259/514] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:09.642060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d8929-82ea-49d9-8c24-47d5b3dd71b1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:09.667032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 256/515] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:09.667396 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83458e-b019-4481-a9e4-9f0ba3597aa2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:09.759891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d8929-82ea-49d9-8c24-47d5b3dd71b1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:09.798444 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83458e-b019-4481-a9e4-9f0ba3597aa2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:09.802987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0ccee-cc29-439a-a2dc-3096cc4ae533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:09.803740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0ccee-cc29-439a-a2dc-3096cc4ae533 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:09.804114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0ccee-cc29-439a-a2dc-3096cc4ae533 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:09.804473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0ccee-cc29-439a-a2dc-3096cc4ae533 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:09.805226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0ccee-cc29-439a-a2dc-3096cc4ae533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:09.805645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0ccee-cc29-439a-a2dc-3096cc4ae533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:09.807169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0ccee-cc29-439a-a2dc-3096cc4ae533 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:09.821377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 260/516] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:09.822257 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0ccee-cc29-439a-a2dc-3096cc4ae533 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:09.844099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016300bc-f2b5-4dca-ad1b-f656985dd7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:09.844923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016300bc-f2b5-4dca-ad1b-f656985dd7a1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:09.845367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016300bc-f2b5-4dca-ad1b-f656985dd7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:09.845724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:09.845953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016300bc-f2b5-4dca-ad1b-f656985dd7a1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:09.857027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016300bc-f2b5-4dca-ad1b-f656985dd7a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:09.859607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016300bc-f2b5-4dca-ad1b-f656985dd7a1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-016300bc-f2b5-4dca-ad1b-f656985dd7a1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:09.874403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:09.875534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:09.876430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 257/517] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:09.877651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016300bc-f2b5-4dca-ad1b-f656985dd7a1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:09.891175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:09.959708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0ccee-cc29-439a-a2dc-3096cc4ae533 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:09.999404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868f827-64ce-43a2-9165-a11406ec6312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:10.000062 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868f827-64ce-43a2-9165-a11406ec6312 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:10.000340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868f827-64ce-43a2-9165-a11406ec6312 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:10.000614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e868f827-64ce-43a2-9165-a11406ec6312 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:10.001310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868f827-64ce-43a2-9165-a11406ec6312 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:10.001612 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868f827-64ce-43a2-9165-a11406ec6312 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:10.003090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e868f827-64ce-43a2-9165-a11406ec6312 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:10.011157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016300bc-f2b5-4dca-ad1b-f656985dd7a1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:10.022062 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 261/518] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:10.022921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868f827-64ce-43a2-9165-a11406ec6312 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:10.073703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4e637-a02c-4742-aea4-0bc4dada8037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:10.074787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4e637-a02c-4742-aea4-0bc4dada8037 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:10.075293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4e637-a02c-4742-aea4-0bc4dada8037 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:10.075795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a4e637-a02c-4742-aea4-0bc4dada8037 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:10.077498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4e637-a02c-4742-aea4-0bc4dada8037 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:10.078245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4e637-a02c-4742-aea4-0bc4dada8037 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:10.080745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a4e637-a02c-4742-aea4-0bc4dada8037 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:10.096338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 258/519] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:10.097200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4e637-a02c-4742-aea4-0bc4dada8037 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:10.118075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:10.143834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e868f827-64ce-43a2-9165-a11406ec6312 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:10.181653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb2ec3-bcbf-4590-8767-a1b49a7f5d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:10.182330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb2ec3-bcbf-4590-8767-a1b49a7f5d9f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:10.182606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb2ec3-bcbf-4590-8767-a1b49a7f5d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:10.182872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb2ec3-bcbf-4590-8767-a1b49a7f5d9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:10.194794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb2ec3-bcbf-4590-8767-a1b49a7f5d9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:10.196298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afb2ec3-bcbf-4590-8767-a1b49a7f5d9f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:10.196462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1afb2ec3-bcbf-4590-8767-a1b49a7f5d9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:10.213726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:10.214840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:10.215769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:10.215769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:10.215769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:10.215769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 262/520] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:10.215769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb2ec3-bcbf-4590-8767-a1b49a7f5d9f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:10.248671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a4e637-a02c-4742-aea4-0bc4dada8037 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:10.288674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e349d-ac83-441d-bd89-df6487b821b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:10.289322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e349d-ac83-441d-bd89-df6487b821b2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:10.289601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e349d-ac83-441d-bd89-df6487b821b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:10.289911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e349d-ac83-441d-bd89-df6487b821b2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:10.299308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e349d-ac83-441d-bd89-df6487b821b2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:10.300799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e349d-ac83-441d-bd89-df6487b821b2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-de8e349d-ac83-441d-bd89-df6487b821b2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:10.314081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:10.321563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 259/521] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:10.323012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e349d-ac83-441d-bd89-df6487b821b2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:10.334634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb2ec3-bcbf-4590-8767-a1b49a7f5d9f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:10.369867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:10.378825 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96721205-a14e-4767-b4a3-f834c4ca7f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:10.379808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96721205-a14e-4767-b4a3-f834c4ca7f82 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:10.380275 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96721205-a14e-4767-b4a3-f834c4ca7f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:10.380750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96721205-a14e-4767-b4a3-f834c4ca7f82 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:10.382021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96721205-a14e-4767-b4a3-f834c4ca7f82 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:10.382543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96721205-a14e-4767-b4a3-f834c4ca7f82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:10.385216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96721205-a14e-4767-b4a3-f834c4ca7f82 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:10.432310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 263/522] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Aug 19 20:55:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:10.433157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96721205-a14e-4767-b4a3-f834c4ca7f82 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:10.442104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e349d-ac83-441d-bd89-df6487b821b2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:10.470452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:10.479637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93269791-d51e-4681-b12d-2c6339ab3c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:10.480251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269791-d51e-4681-b12d-2c6339ab3c63 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:10.480532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269791-d51e-4681-b12d-2c6339ab3c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:10.480813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93269791-d51e-4681-b12d-2c6339ab3c63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:10.481558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93269791-d51e-4681-b12d-2c6339ab3c63 None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=43a9ad5341b54c40a5427045660363c3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:10.481897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93269791-d51e-4681-b12d-2c6339ab3c63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:10.484116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93269791-d51e-4681-b12d-2c6339ab3c63 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:10.490237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 260/523] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:10 2022] GET /identity/v3/role_assignments?user.id=43a9ad5341b54c40a5427045660363c3&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:10.490919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93269791-d51e-4681-b12d-2c6339ab3c63 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:10.561743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96721205-a14e-4767-b4a3-f834c4ca7f82 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:10.606077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f5edc-8871-43b8-b00b-d10a9aa9decd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:10.606703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5edc-8871-43b8-b00b-d10a9aa9decd None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:10.606980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5edc-8871-43b8-b00b-d10a9aa9decd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:10.607255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f5edc-8871-43b8-b00b-d10a9aa9decd None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:10.608738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f5edc-8871-43b8-b00b-d10a9aa9decd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:10.610615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f5edc-8871-43b8-b00b-d10a9aa9decd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:10.637691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 264/524] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:55:10 2022] POST /identity/v3/users => generated 247 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:10.639257 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f5edc-8871-43b8-b00b-d10a9aa9decd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:10.684210 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93269791-d51e-4681-b12d-2c6339ab3c63 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:10.738298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:10.762982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6dda-2455-4403-9a81-d5f36223c627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:10.765021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6dda-2455-4403-9a81-d5f36223c627 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:10.765751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6dda-2455-4403-9a81-d5f36223c627 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:10.766527 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb6dda-2455-4403-9a81-d5f36223c627 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:10.768089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb6dda-2455-4403-9a81-d5f36223c627 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:10.768882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb6dda-2455-4403-9a81-d5f36223c627 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:10.771962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb6dda-2455-4403-9a81-d5f36223c627 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:10.806567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 261/525] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:10.809179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6dda-2455-4403-9a81-d5f36223c627 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:10.814595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f5edc-8871-43b8-b00b-d10a9aa9decd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:10.872644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179a135-c452-4130-9dfe-a46312a4be32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:10.873695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179a135-c452-4130-9dfe-a46312a4be32 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:10.874991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179a135-c452-4130-9dfe-a46312a4be32 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:10.875572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b179a135-c452-4130-9dfe-a46312a4be32 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:10.877000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179a135-c452-4130-9dfe-a46312a4be32 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:10.879582 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179a135-c452-4130-9dfe-a46312a4be32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:10.883553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b179a135-c452-4130-9dfe-a46312a4be32 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:10.886738 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:10.919778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 265/526] 10.222.0.16 () {56 vars in 1139 bytes} [Fri Aug 19 20:55:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:10.921712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179a135-c452-4130-9dfe-a46312a4be32 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:10.954362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb6dda-2455-4403-9a81-d5f36223c627 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:11.000487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17c2e6-6c6d-46fd-9f54-0212d085b6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:11.001605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17c2e6-6c6d-46fd-9f54-0212d085b6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:11.002042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17c2e6-6c6d-46fd-9f54-0212d085b6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:11.003084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa17c2e6-6c6d-46fd-9f54-0212d085b6b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:11.067950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b179a135-c452-4130-9dfe-a46312a4be32 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:11.069779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:11.088661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17c2e6-6c6d-46fd-9f54-0212d085b6b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:11.090190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa17c2e6-6c6d-46fd-9f54-0212d085b6b8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:11.115402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca197e1-be67-4a16-be3b-46ff316ce94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:11.116524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca197e1-be67-4a16-be3b-46ff316ce94f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:11.116999 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca197e1-be67-4a16-be3b-46ff316ce94f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:11.117462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca197e1-be67-4a16-be3b-46ff316ce94f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:11.126650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca197e1-be67-4a16-be3b-46ff316ce94f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:11.128885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca197e1-be67-4a16-be3b-46ff316ce94f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-cca197e1-be67-4a16-be3b-46ff316ce94f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:11.142243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:11.143244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:11.144067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:11.144067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:11.144067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:11.144067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:11.144067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:11.144067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:11.144067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:11.144067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:11.146060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:11.147870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 266/527] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:11.149012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca197e1-be67-4a16-be3b-46ff316ce94f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:11.200458 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 262/528] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:11.201385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17c2e6-6c6d-46fd-9f54-0212d085b6b8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:11.275233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca197e1-be67-4a16-be3b-46ff316ce94f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:11.321212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcca42a-d767-4c0b-ada2-44bcda105bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:11.321930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcca42a-d767-4c0b-ada2-44bcda105bad None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:11.322212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcca42a-d767-4c0b-ada2-44bcda105bad None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:11.322482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcca42a-d767-4c0b-ada2-44bcda105bad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:11.335027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa17c2e6-6c6d-46fd-9f54-0212d085b6b8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:11.363379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:11.394029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ddfa7-917e-4610-8d6c-006caa454d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:11.395568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ddfa7-917e-4610-8d6c-006caa454d45 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:11.396498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ddfa7-917e-4610-8d6c-006caa454d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:11.397427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ddfa7-917e-4610-8d6c-006caa454d45 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:11.399300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ddfa7-917e-4610-8d6c-006caa454d45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:11.400104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ddfa7-917e-4610-8d6c-006caa454d45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:11.403414 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ddfa7-917e-4610-8d6c-006caa454d45 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:11.410463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcca42a-d767-4c0b-ada2-44bcda105bad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:11.411947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcca42a-d767-4c0b-ada2-44bcda105bad None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:11.446455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 263/529] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:11.448529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ddfa7-917e-4610-8d6c-006caa454d45 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:11.465150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:11.554895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 267/530] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:11.556113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcca42a-d767-4c0b-ada2-44bcda105bad None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:11.602372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ddfa7-917e-4610-8d6c-006caa454d45 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:11.644098 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b6924-03a2-4cc7-822c-d33256eb127f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:11.645116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b6924-03a2-4cc7-822c-d33256eb127f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:11.645624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b6924-03a2-4cc7-822c-d33256eb127f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:11.646129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51b6924-03a2-4cc7-822c-d33256eb127f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:11.648938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:11.665727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b6924-03a2-4cc7-822c-d33256eb127f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:11.668867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51b6924-03a2-4cc7-822c-d33256eb127f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c51b6924-03a2-4cc7-822c-d33256eb127f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:11.710724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:11.712505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:11.714232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:11.714232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:11.714232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:11.717385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 264/531] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:11.717977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b6924-03a2-4cc7-822c-d33256eb127f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:11.728422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcca42a-d767-4c0b-ada2-44bcda105bad None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:11.790163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee79338-43a4-44a6-b88f-62ca5913478a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:11.790982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee79338-43a4-44a6-b88f-62ca5913478a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:11.791261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee79338-43a4-44a6-b88f-62ca5913478a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:11.791524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee79338-43a4-44a6-b88f-62ca5913478a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:11.792512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee79338-43a4-44a6-b88f-62ca5913478a None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=54d1ae400b5c4ae0955047b9e7750103) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:11.792766 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee79338-43a4-44a6-b88f-62ca5913478a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:11.794534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee79338-43a4-44a6-b88f-62ca5913478a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:11.803156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 268/532] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:11 2022] GET /identity/v3/role_assignments?user.id=54d1ae400b5c4ae0955047b9e7750103&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:11.804647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee79338-43a4-44a6-b88f-62ca5913478a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:11.844792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:11.926447 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51b6924-03a2-4cc7-822c-d33256eb127f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:12.007889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee79338-43a4-44a6-b88f-62ca5913478a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:12.018622 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d3b5f-b037-4a31-8a7e-555839b71c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:12.019296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d3b5f-b037-4a31-8a7e-555839b71c8d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:12.019584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d3b5f-b037-4a31-8a7e-555839b71c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:12.019863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d3b5f-b037-4a31-8a7e-555839b71c8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:12.054735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d3b5f-b037-4a31-8a7e-555839b71c8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:12.056287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d3b5f-b037-4a31-8a7e-555839b71c8d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:12.074724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-889d3b5f-b037-4a31-8a7e-555839b71c8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:12.081367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:12.082608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:12.083631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:12.083631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:12.083631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:12.083631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 265/533] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:12.095929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c8953-818a-4097-8cd3-f6ad3d20782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:12.096173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d3b5f-b037-4a31-8a7e-555839b71c8d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:12.096340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c8953-818a-4097-8cd3-f6ad3d20782a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:12.096340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c8953-818a-4097-8cd3-f6ad3d20782a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:12.096340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c8953-818a-4097-8cd3-f6ad3d20782a None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:12.096340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c8953-818a-4097-8cd3-f6ad3d20782a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:12.096340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c8953-818a-4097-8cd3-f6ad3d20782a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:12.097572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c8953-818a-4097-8cd3-f6ad3d20782a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:12.126471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 269/534] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:12.135350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c8953-818a-4097-8cd3-f6ad3d20782a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:12.333426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d3b5f-b037-4a31-8a7e-555839b71c8d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:12.362106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:12.402263 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e463c7-a40f-4401-b15b-e8c40d13b896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:12.416848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e463c7-a40f-4401-b15b-e8c40d13b896 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:12.416848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e463c7-a40f-4401-b15b-e8c40d13b896 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:12.416848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e463c7-a40f-4401-b15b-e8c40d13b896 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:12.416848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e463c7-a40f-4401-b15b-e8c40d13b896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:12.416848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e463c7-a40f-4401-b15b-e8c40d13b896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:12.416848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e463c7-a40f-4401-b15b-e8c40d13b896 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:12.430793 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 266/535] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:12.436898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c8953-818a-4097-8cd3-f6ad3d20782a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:12.439653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e463c7-a40f-4401-b15b-e8c40d13b896 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:12.503150 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16800ec-62b2-4707-8443-b74ced848d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:12.503799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16800ec-62b2-4707-8443-b74ced848d6a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:12.504067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16800ec-62b2-4707-8443-b74ced848d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:12.504342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16800ec-62b2-4707-8443-b74ced848d6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:12.523803 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16800ec-62b2-4707-8443-b74ced848d6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:12.526649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16800ec-62b2-4707-8443-b74ced848d6a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b16800ec-62b2-4707-8443-b74ced848d6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:12.571338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:12.573060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:12.575485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:12.575485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:12.575485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:12.575485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:12.575485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:12.575485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:12.575485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:12.575485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:12.575485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 270/536] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 439 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:12.586252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16800ec-62b2-4707-8443-b74ced848d6a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:12.603188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e463c7-a40f-4401-b15b-e8c40d13b896 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:12.709517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be7935-07f6-46e9-b148-3a3ba38d9a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:12.710481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be7935-07f6-46e9-b148-3a3ba38d9a95 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:12.710957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be7935-07f6-46e9-b148-3a3ba38d9a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:12.711428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be7935-07f6-46e9-b148-3a3ba38d9a95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:12.752708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be7935-07f6-46e9-b148-3a3ba38d9a95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:12.754279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be7935-07f6-46e9-b148-3a3ba38d9a95 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:12.763968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-65be7935-07f6-46e9-b148-3a3ba38d9a95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:12.776283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:12.778428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:12.781164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:12.781164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:12.781164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:12.781164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:12.781164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:12.781164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:12.781164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:12.781164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:12.785856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 267/537] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:12.786897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be7935-07f6-46e9-b148-3a3ba38d9a95 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:12.835260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16800ec-62b2-4707-8443-b74ced848d6a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:12.882452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa4864-cce9-4887-b006-4a5546146a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:12.884356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4864-cce9-4887-b006-4a5546146a9c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:12.886950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4864-cce9-4887-b006-4a5546146a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:12.887461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4864-cce9-4887-b006-4a5546146a9c None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:12.888391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa4864-cce9-4887-b006-4a5546146a9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:12.888944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:12.889886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa4864-cce9-4887-b006-4a5546146a9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:12.892669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa4864-cce9-4887-b006-4a5546146a9c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:12.922471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 271/538] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:12.927596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa4864-cce9-4887-b006-4a5546146a9c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:13.024954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be7935-07f6-46e9-b148-3a3ba38d9a95 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:13.075297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f063cc5b-8180-4540-8b62-8cda2a8e04e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:13.077019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063cc5b-8180-4540-8b62-8cda2a8e04e2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:13.077493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063cc5b-8180-4540-8b62-8cda2a8e04e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:13.077993 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f063cc5b-8180-4540-8b62-8cda2a8e04e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:13.079030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063cc5b-8180-4540-8b62-8cda2a8e04e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:13.079922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063cc5b-8180-4540-8b62-8cda2a8e04e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:13.081912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f063cc5b-8180-4540-8b62-8cda2a8e04e2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:13.104499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 268/539] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:13.104499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2a4d0-8f38-4f3d-a3e2-0545c9f240e2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:13.104499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2a4d0-8f38-4f3d-a3e2-0545c9f240e2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:13.104499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2a4d0-8f38-4f3d-a3e2-0545c9f240e2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:13.105309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 269/540] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:13.106949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2a4d0-8f38-4f3d-a3e2-0545c9f240e2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:13.108830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa4864-cce9-4887-b006-4a5546146a9c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:13.121740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:13.165898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002683ed-0041-4ebc-829c-49a5b399b2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:13.166628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002683ed-0041-4ebc-829c-49a5b399b2e0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:13.166911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002683ed-0041-4ebc-829c-49a5b399b2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:13.167188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002683ed-0041-4ebc-829c-49a5b399b2e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:13.168264 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002683ed-0041-4ebc-829c-49a5b399b2e0 None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=b957b367ddf2483f84056ccec57fc515) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:13.168832 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002683ed-0041-4ebc-829c-49a5b399b2e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:13.172225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002683ed-0041-4ebc-829c-49a5b399b2e0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:13.182399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 272/541] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:12 2022] GET /identity/v3/role_assignments?user.id=b957b367ddf2483f84056ccec57fc515&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:13.184415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc094-6925-4942-a997-c91b692b98ef None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:13.184764 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc094-6925-4942-a997-c91b692b98ef None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:13.185041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc094-6925-4942-a997-c91b692b98ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:13.222062 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4c2cc094-6925-4942-a997-c91b692b98ef None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:13.223782 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4c2cc094-6925-4942-a997-c91b692b98ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:13.294092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:13.308222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2a4d0-8f38-4f3d-a3e2-0545c9f240e2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:13.330138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 273/542] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:13.332446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8910d-dd1b-4055-b547-26f5d2089336 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:13.333597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8910d-dd1b-4055-b547-26f5d2089336 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:13.334105 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8910d-dd1b-4055-b547-26f5d2089336 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:13.340846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 274/543] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:13.344738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fecb12-0667-4ef5-a2d0-3acb03c46ef5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:13.345362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fecb12-0667-4ef5-a2d0-3acb03c46ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:13.345871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fecb12-0667-4ef5-a2d0-3acb03c46ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:13.364824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbaa96-985f-4346-908d-9b63b0b5d05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:13.365531 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbaa96-985f-4346-908d-9b63b0b5d05b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:13.365722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbaa96-985f-4346-908d-9b63b0b5d05b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:13.366045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afbaa96-985f-4346-908d-9b63b0b5d05b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:13.366792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbaa96-985f-4346-908d-9b63b0b5d05b None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=b37a4275e7c6441aa9e8bb0298f56589) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:13.367108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbaa96-985f-4346-908d-9b63b0b5d05b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:13.368846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afbaa96-985f-4346-908d-9b63b0b5d05b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:13.371897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-59fecb12-0667-4ef5-a2d0-3acb03c46ef5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:13.375190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-59fecb12-0667-4ef5-a2d0-3acb03c46ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:13.376313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 270/544] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:13 2022] GET /identity/v3/role_assignments?user.id=b37a4275e7c6441aa9e8bb0298f56589&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:13.473117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 275/545] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:13.720037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbaa96-985f-4346-908d-9b63b0b5d05b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:13.875640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afbaa96-985f-4346-908d-9b63b0b5d05b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:13.945805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fecb12-0667-4ef5-a2d0-3acb03c46ef5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:13.964329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244b159-5be7-4b13-9267-4be008ccb220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:13.964985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244b159-5be7-4b13-9267-4be008ccb220 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:13.965261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244b159-5be7-4b13-9267-4be008ccb220 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:13.965539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f244b159-5be7-4b13-9267-4be008ccb220 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:13.986360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f244b159-5be7-4b13-9267-4be008ccb220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f244b159-5be7-4b13-9267-4be008ccb220 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:13.992207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:13.993298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:13.994461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 271/546] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:13.996565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:14.009668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244b159-5be7-4b13-9267-4be008ccb220 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:14.080672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:14.162017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fecb12-0667-4ef5-a2d0-3acb03c46ef5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:14.227316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d45b8d-f561-42ef-9cdd-c04cff29fb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:14.227961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d45b8d-f561-42ef-9cdd-c04cff29fb4f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:14.228239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d45b8d-f561-42ef-9cdd-c04cff29fb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:14.228512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d45b8d-f561-42ef-9cdd-c04cff29fb4f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:14.251377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d45b8d-f561-42ef-9cdd-c04cff29fb4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b9d45b8d-f561-42ef-9cdd-c04cff29fb4f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:14.261723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:14.262695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:14.263521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:14.265529 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 276/547] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:14.271110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d45b8d-f561-42ef-9cdd-c04cff29fb4f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:14.308868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f244b159-5be7-4b13-9267-4be008ccb220 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:14.351416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60215f82-6fe4-4811-b6b7-fd5b509e2471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:14.352034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60215f82-6fe4-4811-b6b7-fd5b509e2471 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:14.352316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60215f82-6fe4-4811-b6b7-fd5b509e2471 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:14.352600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60215f82-6fe4-4811-b6b7-fd5b509e2471 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:14.353279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60215f82-6fe4-4811-b6b7-fd5b509e2471 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:14.353584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60215f82-6fe4-4811-b6b7-fd5b509e2471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:14.355003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60215f82-6fe4-4811-b6b7-fd5b509e2471 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:14.381281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 272/548] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:14.387345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60215f82-6fe4-4811-b6b7-fd5b509e2471 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:14.430223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:14.447646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d45b8d-f561-42ef-9cdd-c04cff29fb4f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:14.536612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:14.539578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151c263-f86e-422a-bd67-a337c7d1b70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:14.540231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151c263-f86e-422a-bd67-a337c7d1b70c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:14.540483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151c263-f86e-422a-bd67-a337c7d1b70c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:14.540756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6151c263-f86e-422a-bd67-a337c7d1b70c None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:14.541443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151c263-f86e-422a-bd67-a337c7d1b70c None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:14.541756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151c263-f86e-422a-bd67-a337c7d1b70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:14.543292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6151c263-f86e-422a-bd67-a337c7d1b70c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:14.553610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 277/549] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:14.559518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151c263-f86e-422a-bd67-a337c7d1b70c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:14.568965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60215f82-6fe4-4811-b6b7-fd5b509e2471 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:14.609122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac185e30-4a6e-44d0-8bce-414aa8f87a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:14.609785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac185e30-4a6e-44d0-8bce-414aa8f87a1a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:14.610071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac185e30-4a6e-44d0-8bce-414aa8f87a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:14.610344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac185e30-4a6e-44d0-8bce-414aa8f87a1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:14.635756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac185e30-4a6e-44d0-8bce-414aa8f87a1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:14.637239 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac185e30-4a6e-44d0-8bce-414aa8f87a1a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ac185e30-4a6e-44d0-8bce-414aa8f87a1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:14.663348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:14.664408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:14.665264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:14.665264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:14.665264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:14.667678 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 273/550] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:14.674937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac185e30-4a6e-44d0-8bce-414aa8f87a1a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:14.729693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6151c263-f86e-422a-bd67-a337c7d1b70c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:14.780729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4deae-45ad-40e6-a54e-9e93460432b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:14.781700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4deae-45ad-40e6-a54e-9e93460432b1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:14.782005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4deae-45ad-40e6-a54e-9e93460432b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:14.782762 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4deae-45ad-40e6-a54e-9e93460432b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:14.796340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4deae-45ad-40e6-a54e-9e93460432b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:14.797927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4deae-45ad-40e6-a54e-9e93460432b1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-12e4deae-45ad-40e6-a54e-9e93460432b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:14.814736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:14.815874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:14.816736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:14.816736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:14.816736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:14.816736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 278/551] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:14.821584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737894f-62a8-4123-9a66-42a351031604 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:14.821584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737894f-62a8-4123-9a66-42a351031604 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:14.821584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c737894f-62a8-4123-9a66-42a351031604 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:14.823815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 279/552] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:14.824701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737894f-62a8-4123-9a66-42a351031604 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:14.830010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:14.845688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac185e30-4a6e-44d0-8bce-414aa8f87a1a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:14.886607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:14.893360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b77a9c-c57f-4ba7-be9d-3c19df65bc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:14.894018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b77a9c-c57f-4ba7-be9d-3c19df65bc4b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:14.894297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b77a9c-c57f-4ba7-be9d-3c19df65bc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:14.894630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b77a9c-c57f-4ba7-be9d-3c19df65bc4b None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:14.895401 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b77a9c-c57f-4ba7-be9d-3c19df65bc4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:14.895653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b77a9c-c57f-4ba7-be9d-3c19df65bc4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:14.897103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b77a9c-c57f-4ba7-be9d-3c19df65bc4b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:14.911050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 274/553] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:14.917557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12203b-b6bb-4f67-8ed9-af77e6de4c28 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:14.917877 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12203b-b6bb-4f67-8ed9-af77e6de4c28 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:14.918164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c12203b-b6bb-4f67-8ed9-af77e6de4c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:14.937877 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c12203b-b6bb-4f67-8ed9-af77e6de4c28 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:14.939339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c12203b-b6bb-4f67-8ed9-af77e6de4c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:14.965286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c737894f-62a8-4123-9a66-42a351031604 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:15.004953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7451b-2ebd-4742-b0ff-ae258e6b53f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:15.005954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7451b-2ebd-4742-b0ff-ae258e6b53f4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:15.006511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7451b-2ebd-4742-b0ff-ae258e6b53f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:15.007132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7451b-2ebd-4742-b0ff-ae258e6b53f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:15.008076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7451b-2ebd-4742-b0ff-ae258e6b53f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:15.008583 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7451b-2ebd-4742-b0ff-ae258e6b53f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:15.010295 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7451b-2ebd-4742-b0ff-ae258e6b53f4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:15.025451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 280/554] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:15.026887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7451b-2ebd-4742-b0ff-ae258e6b53f4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:15.032060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 275/555] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:15.033389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12203b-b6bb-4f67-8ed9-af77e6de4c28 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:15.106084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:15.166555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c12203b-b6bb-4f67-8ed9-af77e6de4c28 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:15.187884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7451b-2ebd-4742-b0ff-ae258e6b53f4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:15.194542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:15.209028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf95cf8-04f4-4976-aead-68c372c8cb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:15.209682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf95cf8-04f4-4976-aead-68c372c8cb01 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:15.209995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf95cf8-04f4-4976-aead-68c372c8cb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:15.210272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf95cf8-04f4-4976-aead-68c372c8cb01 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:15.219824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf95cf8-04f4-4976-aead-68c372c8cb01 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:15.221448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf95cf8-04f4-4976-aead-68c372c8cb01 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8bf95cf8-04f4-4976-aead-68c372c8cb01 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:15.235412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:15.236415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:15.237720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 276/556] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:15.251129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf95cf8-04f4-4976-aead-68c372c8cb01 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:15.272627 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3679d-5b40-453d-9999-da6c5d49df5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:15.274186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3679d-5b40-453d-9999-da6c5d49df5c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:15.274462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3679d-5b40-453d-9999-da6c5d49df5c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:15.274740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3679d-5b40-453d-9999-da6c5d49df5c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:15.292417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3679d-5b40-453d-9999-da6c5d49df5c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:15.295793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3679d-5b40-453d-9999-da6c5d49df5c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bea3679d-5b40-453d-9999-da6c5d49df5c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:15.319156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:15.320159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:15.322211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 281/557] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Aug 19 20:55:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:15.326823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3679d-5b40-453d-9999-da6c5d49df5c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:15.433480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf95cf8-04f4-4976-aead-68c372c8cb01 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:15.454708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3679d-5b40-453d-9999-da6c5d49df5c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:15.481446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:15.489286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634dc9d-b3df-42b7-b7b9-59bb3cd0df5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:15.490504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634dc9d-b3df-42b7-b7b9-59bb3cd0df5d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:15.490960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634dc9d-b3df-42b7-b7b9-59bb3cd0df5d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:15.491377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634dc9d-b3df-42b7-b7b9-59bb3cd0df5d None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:15.492284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634dc9d-b3df-42b7-b7b9-59bb3cd0df5d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:15.492751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634dc9d-b3df-42b7-b7b9-59bb3cd0df5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:15.494504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634dc9d-b3df-42b7-b7b9-59bb3cd0df5d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:15.494934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c93459-7df3-4b65-a31d-a295bf982f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:15.495706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c93459-7df3-4b65-a31d-a295bf982f65 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:15.496198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c93459-7df3-4b65-a31d-a295bf982f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:15.496624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c93459-7df3-4b65-a31d-a295bf982f65 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:15.497512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c93459-7df3-4b65-a31d-a295bf982f65 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:15.498411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c93459-7df3-4b65-a31d-a295bf982f65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:15.500897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c93459-7df3-4b65-a31d-a295bf982f65 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:15.518512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 277/558] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Aug 19 20:55:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:15.519488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634dc9d-b3df-42b7-b7b9-59bb3cd0df5d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:15.525890 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 282/559] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Aug 19 20:55:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:15.527071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c93459-7df3-4b65-a31d-a295bf982f65 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:15.578375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:15.647811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c93459-7df3-4b65-a31d-a295bf982f65 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:15.662883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634dc9d-b3df-42b7-b7b9-59bb3cd0df5d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:15.691652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f1489-db7e-478e-82d4-9fabeeac7bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:15.692179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f1489-db7e-478e-82d4-9fabeeac7bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:15.692468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f1489-db7e-478e-82d4-9fabeeac7bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:15.692735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f1489-db7e-478e-82d4-9fabeeac7bd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:15.702008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f00da-d4e1-4e94-8604-feb76b51ee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:15.702912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f00da-d4e1-4e94-8604-feb76b51ee8f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:15.703188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f00da-d4e1-4e94-8604-feb76b51ee8f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:15.703467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f00da-d4e1-4e94-8604-feb76b51ee8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:15.716823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f00da-d4e1-4e94-8604-feb76b51ee8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-390f00da-d4e1-4e94-8604-feb76b51ee8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:15.722371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:15.724145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 278/560] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:15.726407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f00da-d4e1-4e94-8604-feb76b51ee8f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:15.759416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:15.782522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f1489-db7e-478e-82d4-9fabeeac7bd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:15.784034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f1489-db7e-478e-82d4-9fabeeac7bd6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:15.858892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:15.862900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f00da-d4e1-4e94-8604-feb76b51ee8f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:15.908352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7a13c-08e7-4449-a4f1-624c260aecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:15.908983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7a13c-08e7-4449-a4f1-624c260aecb3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:15.909259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7a13c-08e7-4449-a4f1-624c260aecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:15.909531 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7a13c-08e7-4449-a4f1-624c260aecb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:15.914708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 283/561] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:15.914708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bace89-2e51-4233-a7d1-ecdf721ae566 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:15.914708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bace89-2e51-4233-a7d1-ecdf721ae566 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:15.914708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bace89-2e51-4233-a7d1-ecdf721ae566 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:15.914708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 284/562] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:15.915180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bace89-2e51-4233-a7d1-ecdf721ae566 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:15.924939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7a13c-08e7-4449-a4f1-624c260aecb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:15.926461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7a13c-08e7-4449-a4f1-624c260aecb3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-cac7a13c-08e7-4449-a4f1-624c260aecb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:15.942550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:15.945863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:15.946857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:15.946857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:15.946857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:15.946857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 279/563] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:15.946857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7a13c-08e7-4449-a4f1-624c260aecb3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:16.023947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:16.047465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bace89-2e51-4233-a7d1-ecdf721ae566 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:16.057600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7a13c-08e7-4449-a4f1-624c260aecb3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:16.086335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d64adb-c80a-4c0a-bd29-f79ecee43eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:16.086989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d64adb-c80a-4c0a-bd29-f79ecee43eba None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.087273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d64adb-c80a-4c0a-bd29-f79ecee43eba None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.087549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d64adb-c80a-4c0a-bd29-f79ecee43eba None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.088225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d64adb-c80a-4c0a-bd29-f79ecee43eba None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:16.088519 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d64adb-c80a-4c0a-bd29-f79ecee43eba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:16.089925 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d64adb-c80a-4c0a-bd29-f79ecee43eba None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:16.093187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38883b-f618-4690-9e7a-c4a58a4731bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:16.093859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38883b-f618-4690-9e7a-c4a58a4731bf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.094137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38883b-f618-4690-9e7a-c4a58a4731bf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.094409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38883b-f618-4690-9e7a-c4a58a4731bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.099542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 285/564] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:16.100786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1c0b5-7692-4c54-9400-f7739133ac3f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.101049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1c0b5-7692-4c54-9400-f7739133ac3f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.101320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1c0b5-7692-4c54-9400-f7739133ac3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.105740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38883b-f618-4690-9e7a-c4a58a4731bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:16.107240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38883b-f618-4690-9e7a-c4a58a4731bf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:16.119971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8eb1c0b5-7692-4c54-9400-f7739133ac3f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:16.121175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8eb1c0b5-7692-4c54-9400-f7739133ac3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7d38883b-f618-4690-9e7a-c4a58a4731bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:16.122296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:16.123526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:16.124804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:16.124804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:16.124804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:16.124804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 280/565] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:16.124804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38883b-f618-4690-9e7a-c4a58a4731bf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:16.128339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:16.198700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 286/566] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:16.199894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547a22e-2450-4307-868a-665ca995420d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.200182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547a22e-2450-4307-868a-665ca995420d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.200459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547a22e-2450-4307-868a-665ca995420d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.202353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 287/567] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:16.203286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547a22e-2450-4307-868a-665ca995420d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:16.258618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38883b-f618-4690-9e7a-c4a58a4731bf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:16.305250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:16.313586 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90400f6-15a0-4009-8092-5f67a06c8c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:16.314519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90400f6-15a0-4009-8092-5f67a06c8c44 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.314903 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90400f6-15a0-4009-8092-5f67a06c8c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.315365 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90400f6-15a0-4009-8092-5f67a06c8c44 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.316468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90400f6-15a0-4009-8092-5f67a06c8c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:16.316972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90400f6-15a0-4009-8092-5f67a06c8c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:16.319292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90400f6-15a0-4009-8092-5f67a06c8c44 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:16.339096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 281/568] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:16.340507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90400f6-15a0-4009-8092-5f67a06c8c44 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:16.356340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547a22e-2450-4307-868a-665ca995420d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:16.399265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d0a1c-c199-47ae-b372-943bc12a1fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:16.400020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d0a1c-c199-47ae-b372-943bc12a1fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.400365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d0a1c-c199-47ae-b372-943bc12a1fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.400706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d0a1c-c199-47ae-b372-943bc12a1fc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.413248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d0a1c-c199-47ae-b372-943bc12a1fc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:16.414491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:16.415061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d0a1c-c199-47ae-b372-943bc12a1fc0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5d9d0a1c-c199-47ae-b372-943bc12a1fc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:16.432440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:16.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:16.434843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:16.434843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:16.434843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:16.437962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 288/569] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:16.437962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc01db-b681-47d8-bf14-57281db3f0d6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.437962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc01db-b681-47d8-bf14-57281db3f0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.437962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cc01db-b681-47d8-bf14-57281db3f0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.457338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-20cc01db-b681-47d8-bf14-57281db3f0d6 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:16.458488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-20cc01db-b681-47d8-bf14-57281db3f0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:16.461393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90400f6-15a0-4009-8092-5f67a06c8c44 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:16.500409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1589e-5f85-417d-835a-dba0e9c68c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:16.501137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1589e-5f85-417d-835a-dba0e9c68c2f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.501419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1589e-5f85-417d-835a-dba0e9c68c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.501697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1589e-5f85-417d-835a-dba0e9c68c2f None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.502504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1589e-5f85-417d-835a-dba0e9c68c2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:16.502821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1589e-5f85-417d-835a-dba0e9c68c2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:16.504266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1589e-5f85-417d-835a-dba0e9c68c2f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:16.521240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 282/570] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:16.521240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1589e-5f85-417d-835a-dba0e9c68c2f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:16.540400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 289/571] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:16.541261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc01db-b681-47d8-bf14-57281db3f0d6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:16.603280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:16.654278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1589e-5f85-417d-835a-dba0e9c68c2f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:16.664704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cc01db-b681-47d8-bf14-57281db3f0d6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:16.695912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:16.708973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51e461-428f-4624-be84-36afa4efac15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:16.709504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51e461-428f-4624-be84-36afa4efac15 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.709819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51e461-428f-4624-be84-36afa4efac15 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.710085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51e461-428f-4624-be84-36afa4efac15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.715261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0665d-5082-4ccf-bedd-3ce73910f5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:16.716324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0665d-5082-4ccf-bedd-3ce73910f5ba None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.717345 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0665d-5082-4ccf-bedd-3ce73910f5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.717894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f0665d-5082-4ccf-bedd-3ce73910f5ba None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.718252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51e461-428f-4624-be84-36afa4efac15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:16.719103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0665d-5082-4ccf-bedd-3ce73910f5ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:16.719558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0665d-5082-4ccf-bedd-3ce73910f5ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:16.719708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51e461-428f-4624-be84-36afa4efac15 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:16.720990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f0665d-5082-4ccf-bedd-3ce73910f5ba None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0c51e461-428f-4624-be84-36afa4efac15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:16.733454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:16.735278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:16.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:16.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:16.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:16.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:16.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:16.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:16.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:16.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:16.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 283/572] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:16.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51e461-428f-4624-be84-36afa4efac15 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:16.750467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 290/573] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:16.750467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0665d-5082-4ccf-bedd-3ce73910f5ba None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:16.872813 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f0665d-5082-4ccf-bedd-3ce73910f5ba None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:16.874855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51e461-428f-4624-be84-36afa4efac15 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:16.894921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:16.912880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789684c-edd5-437a-bd89-3d59b4ad2735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:16.913504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789684c-edd5-437a-bd89-3d59b4ad2735 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.913809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789684c-edd5-437a-bd89-3d59b4ad2735 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.914084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6789684c-edd5-437a-bd89-3d59b4ad2735 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.922872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdd951-17c7-4075-b379-6e20a42eeaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:16.923199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdd951-17c7-4075-b379-6e20a42eeaef None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:16.923512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdd951-17c7-4075-b379-6e20a42eeaef None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:16.923872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bdd951-17c7-4075-b379-6e20a42eeaef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:16.926832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6789684c-edd5-437a-bd89-3d59b4ad2735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6789684c-edd5-437a-bd89-3d59b4ad2735 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:16.931962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:16.933026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:16.934890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:16.935643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bdd951-17c7-4075-b379-6e20a42eeaef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:16.936403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 284/574] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:16.936529 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bdd951-17c7-4075-b379-6e20a42eeaef None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:16.936669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789684c-edd5-437a-bd89-3d59b4ad2735 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-43bdd951-17c7-4075-b379-6e20a42eeaef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:16.948458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:16.949544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:16.950496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:16.950496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:16.950496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:16.950496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:16.950496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:16.950496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:16.950496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:16.950496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:16.950496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 291/575] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:16.950983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdd951-17c7-4075-b379-6e20a42eeaef None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:16.990758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:17.058843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6789684c-edd5-437a-bd89-3d59b4ad2735 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:17.077883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bdd951-17c7-4075-b379-6e20a42eeaef None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:17.108352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3442f-240c-480c-992a-9b1c06f11a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:17.109177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3442f-240c-480c-992a-9b1c06f11a9c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:17.109459 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3442f-240c-480c-992a-9b1c06f11a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:17.109733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b3442f-240c-480c-992a-9b1c06f11a9c None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:17.110810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3442f-240c-480c-992a-9b1c06f11a9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:17.111353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3442f-240c-480c-992a-9b1c06f11a9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:17.113523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b3442f-240c-480c-992a-9b1c06f11a9c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:17.118446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f01730-e74e-46dc-b2a7-6e87d1415871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:17.119349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f01730-e74e-46dc-b2a7-6e87d1415871 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:17.119632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f01730-e74e-46dc-b2a7-6e87d1415871 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:17.119906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f01730-e74e-46dc-b2a7-6e87d1415871 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:17.128775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f01730-e74e-46dc-b2a7-6e87d1415871 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:17.130280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f01730-e74e-46dc-b2a7-6e87d1415871 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:17.132386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 285/576] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:17.133208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3442f-240c-480c-992a-9b1c06f11a9c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-11f01730-e74e-46dc-b2a7-6e87d1415871 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:17.143102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:17.144236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 292/577] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:17.149964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f01730-e74e-46dc-b2a7-6e87d1415871 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:17.193837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:17.246337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b3442f-240c-480c-992a-9b1c06f11a9c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:17.249858 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:17.272841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f01730-e74e-46dc-b2a7-6e87d1415871 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:17.288410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89a886-0c3e-49b0-8095-bf82141ae809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:17.293243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89a886-0c3e-49b0-8095-bf82141ae809 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:17.293243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89a886-0c3e-49b0-8095-bf82141ae809 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:17.293243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89a886-0c3e-49b0-8095-bf82141ae809 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:17.303931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89a886-0c3e-49b0-8095-bf82141ae809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:17.316059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928bdf1-83c7-4a78-95c0-7fbbfdaf75d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:17.316663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928bdf1-83c7-4a78-95c0-7fbbfdaf75d6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:17.316940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928bdf1-83c7-4a78-95c0-7fbbfdaf75d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:17.317204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b928bdf1-83c7-4a78-95c0-7fbbfdaf75d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ca89a886-0c3e-49b0-8095-bf82141ae809 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:17.317400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:17.318507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:17.319363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:17.319978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928bdf1-83c7-4a78-95c0-7fbbfdaf75d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:17.319978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928bdf1-83c7-4a78-95c0-7fbbfdaf75d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:17.320100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 286/578] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:17.320166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b928bdf1-83c7-4a78-95c0-7fbbfdaf75d6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:17.320241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89a886-0c3e-49b0-8095-bf82141ae809 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:17.329331 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 293/579] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:17.330205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928bdf1-83c7-4a78-95c0-7fbbfdaf75d6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:17.437544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89a886-0c3e-49b0-8095-bf82141ae809 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:17.449639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:17.455400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b928bdf1-83c7-4a78-95c0-7fbbfdaf75d6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:17.476110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b612b1-e2e8-45ff-ac6c-56aff3c7cfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:17.476714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b612b1-e2e8-45ff-ac6c-56aff3c7cfbc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:17.476988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b612b1-e2e8-45ff-ac6c-56aff3c7cfbc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:17.477259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b612b1-e2e8-45ff-ac6c-56aff3c7cfbc None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:17.478005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b612b1-e2e8-45ff-ac6c-56aff3c7cfbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:17.478300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b612b1-e2e8-45ff-ac6c-56aff3c7cfbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:17.479741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b612b1-e2e8-45ff-ac6c-56aff3c7cfbc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:17.493986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4ff2-aeb5-4c11-9045-947c75680a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:17.500206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 287/580] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:17.500408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4ff2-aeb5-4c11-9045-947c75680a68 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:17.500408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4ff2-aeb5-4c11-9045-947c75680a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:17.500408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee4ff2-aeb5-4c11-9045-947c75680a68 None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/43a9ad5341b54c40a5427045660363c3/roles/a797bb15d18b42148f36a77bd6d59606` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:17.500635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b612b1-e2e8-45ff-ac6c-56aff3c7cfbc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:17.513291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee4ff2-aeb5-4c11-9045-947c75680a68 None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=43a9ad5341b54c40a5427045660363c3, role_id=a797bb15d18b42148f36a77bd6d59606)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:17.515504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee4ff2-aeb5-4c11-9045-947c75680a68 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:17.530920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 294/581] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:55:17 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/43a9ad5341b54c40a5427045660363c3/roles/a797bb15d18b42148f36a77bd6d59606 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:55:17.531769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4ff2-aeb5-4c11-9045-947c75680a68 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:17.534081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:17.646788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b612b1-e2e8-45ff-ac6c-56aff3c7cfbc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:17.683299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee4ff2-aeb5-4c11-9045-947c75680a68 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:17.693325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4919885-e4e0-41a8-8fa5-50bb122096e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:17.693987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4919885-e4e0-41a8-8fa5-50bb122096e3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:17.694268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4919885-e4e0-41a8-8fa5-50bb122096e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:17.694541 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4919885-e4e0-41a8-8fa5-50bb122096e3 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:17.695294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4919885-e4e0-41a8-8fa5-50bb122096e3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:17.695601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4919885-e4e0-41a8-8fa5-50bb122096e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:17.697207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4919885-e4e0-41a8-8fa5-50bb122096e3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:17.724503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 288/582] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Aug 19 20:55:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:17.726034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4919885-e4e0-41a8-8fa5-50bb122096e3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:17.744340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdefbe-31e7-4561-9ed9-225545be77ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:17.744955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdefbe-31e7-4561-9ed9-225545be77ee None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:17.745232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdefbe-31e7-4561-9ed9-225545be77ee None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:17.745491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdefbe-31e7-4561-9ed9-225545be77ee None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:17.746235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdefbe-31e7-4561-9ed9-225545be77ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:17.746537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdefbe-31e7-4561-9ed9-225545be77ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:17.747922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdefbe-31e7-4561-9ed9-225545be77ee None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:17.757571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 295/583] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:17.758267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdefbe-31e7-4561-9ed9-225545be77ee None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:17.786018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:17.836662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:17.872826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4919885-e4e0-41a8-8fa5-50bb122096e3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:17.904960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdefbe-31e7-4561-9ed9-225545be77ee None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:17.939604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9f6b4-e240-45f6-9db4-fb7d0f6bbde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:17.940464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f6b4-e240-45f6-9db4-fb7d0f6bbde7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:17.940864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f6b4-e240-45f6-9db4-fb7d0f6bbde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:17.941168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f6b4-e240-45f6-9db4-fb7d0f6bbde7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:17.945890 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71913725-6077-4394-b325-9cd1032365f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:17.946509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71913725-6077-4394-b325-9cd1032365f1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:17.946791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71913725-6077-4394-b325-9cd1032365f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:17.947068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71913725-6077-4394-b325-9cd1032365f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:17.947797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71913725-6077-4394-b325-9cd1032365f1 None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=61a0906b8b134c178b4dece2891f1596) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:17.948162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71913725-6077-4394-b325-9cd1032365f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:17.949682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71913725-6077-4394-b325-9cd1032365f1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:17.953416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9f6b4-e240-45f6-9db4-fb7d0f6bbde7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:17.954910 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9f6b4-e240-45f6-9db4-fb7d0f6bbde7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:17.966186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 296/584] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:17 2022] GET /identity/v3/role_assignments?user.id=61a0906b8b134c178b4dece2891f1596&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:17.966186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71913725-6077-4394-b325-9cd1032365f1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f0e9f6b4-e240-45f6-9db4-fb7d0f6bbde7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:17.985313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:17.986470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:17.987509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:17.987509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:17.987509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:17.987962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 289/585] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:17.988900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9f6b4-e240-45f6-9db4-fb7d0f6bbde7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:18.082728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:18.123899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71913725-6077-4394-b325-9cd1032365f1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:18.155277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9f6b4-e240-45f6-9db4-fb7d0f6bbde7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:18.187292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3bf07-28e6-4278-889c-1615c17ea163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:18.188841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3bf07-28e6-4278-889c-1615c17ea163 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:18.189576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3bf07-28e6-4278-889c-1615c17ea163 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:18.190101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3bf07-28e6-4278-889c-1615c17ea163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:18.206559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3bf07-28e6-4278-889c-1615c17ea163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:18.208298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3bf07-28e6-4278-889c-1615c17ea163 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:18.208568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199c592-57fc-4224-8224-8ea6eb8ab0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:18.209567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199c592-57fc-4224-8224-8ea6eb8ab0f8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:18.210275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199c592-57fc-4224-8224-8ea6eb8ab0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:18.210806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199c592-57fc-4224-8224-8ea6eb8ab0f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:18.220581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7eb3bf07-28e6-4278-889c-1615c17ea163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:18.225541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:18.227339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:18.228223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:18.228223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:18.228223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:18.228869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199c592-57fc-4224-8224-8ea6eb8ab0f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:18.229913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 297/586] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:18.230926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199c592-57fc-4224-8224-8ea6eb8ab0f8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:18.231035 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3bf07-28e6-4278-889c-1615c17ea163 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c199c592-57fc-4224-8224-8ea6eb8ab0f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:18.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:18.256680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:18.257659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:18.257659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:18.257659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:18.263913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 290/587] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:18.263913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199c592-57fc-4224-8224-8ea6eb8ab0f8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:18.357482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3bf07-28e6-4278-889c-1615c17ea163 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:18.402165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:18.412096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ecd5d-aee2-47e3-971d-19c1ff5a0809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:18.412720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ecd5d-aee2-47e3-971d-19c1ff5a0809 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:18.413006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ecd5d-aee2-47e3-971d-19c1ff5a0809 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:18.413288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ecd5d-aee2-47e3-971d-19c1ff5a0809 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:18.414186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ecd5d-aee2-47e3-971d-19c1ff5a0809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:18.414496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ecd5d-aee2-47e3-971d-19c1ff5a0809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:18.416047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ecd5d-aee2-47e3-971d-19c1ff5a0809 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:18.418859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199c592-57fc-4224-8224-8ea6eb8ab0f8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:18.429933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 298/588] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:18.431714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ecd5d-aee2-47e3-971d-19c1ff5a0809 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:18.457698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d6db-1081-457b-b3e4-8c2167f9bc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:18.458346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d6db-1081-457b-b3e4-8c2167f9bc4a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:18.458624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d6db-1081-457b-b3e4-8c2167f9bc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:18.458893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c5d6db-1081-457b-b3e4-8c2167f9bc4a None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:18.459555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5d6db-1081-457b-b3e4-8c2167f9bc4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:18.459858 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5d6db-1081-457b-b3e4-8c2167f9bc4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:18.461261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c5d6db-1081-457b-b3e4-8c2167f9bc4a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:18.475772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 291/589] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:18.476657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d6db-1081-457b-b3e4-8c2167f9bc4a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:18.538544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:18.550207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ecd5d-aee2-47e3-971d-19c1ff5a0809 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:18.597327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c5d6db-1081-457b-b3e4-8c2167f9bc4a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:18.600715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60120e1d-719b-4873-a524-6a50a541d3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:18.601586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60120e1d-719b-4873-a524-6a50a541d3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:18.602073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60120e1d-719b-4873-a524-6a50a541d3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:18.602515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60120e1d-719b-4873-a524-6a50a541d3e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:18.603421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60120e1d-719b-4873-a524-6a50a541d3e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:18.603946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60120e1d-719b-4873-a524-6a50a541d3e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:18.605652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60120e1d-719b-4873-a524-6a50a541d3e7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:18.623538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 299/590] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:18.624973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60120e1d-719b-4873-a524-6a50a541d3e7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:18.640137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ccf89-93d7-45c9-bdd4-0c7e1e8c16a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:18.641251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ccf89-93d7-45c9-bdd4-0c7e1e8c16a0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:18.641251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ccf89-93d7-45c9-bdd4-0c7e1e8c16a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:18.641543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ccf89-93d7-45c9-bdd4-0c7e1e8c16a0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:18.650249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ccf89-93d7-45c9-bdd4-0c7e1e8c16a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:18.651974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ccf89-93d7-45c9-bdd4-0c7e1e8c16a0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e58ccf89-93d7-45c9-bdd4-0c7e1e8c16a0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:18.666741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:18.667780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 292/591] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:18.668684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ccf89-93d7-45c9-bdd4-0c7e1e8c16a0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:18.697724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:18.770767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60120e1d-719b-4873-a524-6a50a541d3e7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:18.792452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ccf89-93d7-45c9-bdd4-0c7e1e8c16a0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:18.803207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:18.819709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359caa70-3806-42bf-9995-3f18b7d6af95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:18.820738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359caa70-3806-42bf-9995-3f18b7d6af95 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:18.821162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359caa70-3806-42bf-9995-3f18b7d6af95 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:18.821580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359caa70-3806-42bf-9995-3f18b7d6af95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:18.830423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359caa70-3806-42bf-9995-3f18b7d6af95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:18.832267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359caa70-3806-42bf-9995-3f18b7d6af95 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:18.836790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153905d8-5bb6-414a-ac80-1f3c6657b089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:18.837543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153905d8-5bb6-414a-ac80-1f3c6657b089 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:18.837978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153905d8-5bb6-414a-ac80-1f3c6657b089 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:18.838348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153905d8-5bb6-414a-ac80-1f3c6657b089 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-359caa70-3806-42bf-9995-3f18b7d6af95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:18.845857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:18.846811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:18.847691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:18.847691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:18.847691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:18.847691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:18.847691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:18.847691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:18.847691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:18.847691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:18.848113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153905d8-5bb6-414a-ac80-1f3c6657b089 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:18.849074 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153905d8-5bb6-414a-ac80-1f3c6657b089 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:18.854030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 300/592] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:18.855121 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359caa70-3806-42bf-9995-3f18b7d6af95 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-153905d8-5bb6-414a-ac80-1f3c6657b089 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:18.862618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:18.863628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:18.864537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:18.864537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:18.864537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:18.864537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:18.864537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:18.864537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:18.864537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:18.864537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:18.864537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 20:55:18.864537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:18.868555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 293/593] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:18.870064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153905d8-5bb6-414a-ac80-1f3c6657b089 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:18.975785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359caa70-3806-42bf-9995-3f18b7d6af95 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:18.981496 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153905d8-5bb6-414a-ac80-1f3c6657b089 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:18.985133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:19.021125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf568e-9f46-4ba8-ab90-9c9e67013d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:19.021792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf568e-9f46-4ba8-ab90-9c9e67013d28 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:19.022087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf568e-9f46-4ba8-ab90-9c9e67013d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:19.022361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf568e-9f46-4ba8-ab90-9c9e67013d28 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:19.023314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf568e-9f46-4ba8-ab90-9c9e67013d28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:19.023445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf568e-9f46-4ba8-ab90-9c9e67013d28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:19.024859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf568e-9f46-4ba8-ab90-9c9e67013d28 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:19.034044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549a605-d7e5-410a-b605-774dde38c5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:19.035120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549a605-d7e5-410a-b605-774dde38c5ed None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:19.035605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549a605-d7e5-410a-b605-774dde38c5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:19.036081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549a605-d7e5-410a-b605-774dde38c5ed None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:19.037738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549a605-d7e5-410a-b605-774dde38c5ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:19.038610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549a605-d7e5-410a-b605-774dde38c5ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:19.040982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549a605-d7e5-410a-b605-774dde38c5ed None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:19.047125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 294/594] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:19.047125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf568e-9f46-4ba8-ab90-9c9e67013d28 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:19.076321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 301/595] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Aug 19 20:55:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:19.077862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549a605-d7e5-410a-b605-774dde38c5ed None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:19.079474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:19.204145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf568e-9f46-4ba8-ab90-9c9e67013d28 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:19.254900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549a605-d7e5-410a-b605-774dde38c5ed None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:19.265547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1c826-2500-496a-a74e-b15545f9ec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:19.266257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1c826-2500-496a-a74e-b15545f9ec57 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:19.266615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1c826-2500-496a-a74e-b15545f9ec57 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:19.266964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1c826-2500-496a-a74e-b15545f9ec57 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:19.267792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1c826-2500-496a-a74e-b15545f9ec57 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:19.268172 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1c826-2500-496a-a74e-b15545f9ec57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:19.270070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1c826-2500-496a-a74e-b15545f9ec57 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:19.296661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 295/596] 10.222.0.16 () {56 vars in 1139 bytes} [Fri Aug 19 20:55:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:19.297793 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1c826-2500-496a-a74e-b15545f9ec57 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:19.301409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9feb2-408b-413d-aca6-09c0bac9fca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:19.302381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9feb2-408b-413d-aca6-09c0bac9fca5 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:19.303717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9feb2-408b-413d-aca6-09c0bac9fca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:19.304172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9feb2-408b-413d-aca6-09c0bac9fca5 None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/54d1ae400b5c4ae0955047b9e7750103/roles/a797bb15d18b42148f36a77bd6d59606` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:19.324197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9feb2-408b-413d-aca6-09c0bac9fca5 None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=54d1ae400b5c4ae0955047b9e7750103, role_id=a797bb15d18b42148f36a77bd6d59606)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:19.325986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9feb2-408b-413d-aca6-09c0bac9fca5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:19.353796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 302/597] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:55:19 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/54d1ae400b5c4ae0955047b9e7750103/roles/a797bb15d18b42148f36a77bd6d59606 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:55:19.353796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9feb2-408b-413d-aca6-09c0bac9fca5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:19.397463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:19.507069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:19.541959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1c826-2500-496a-a74e-b15545f9ec57 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:19.581644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9feb2-408b-413d-aca6-09c0bac9fca5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:19.612968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812f16f-c413-47b0-9028-c360025554fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:19.613707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812f16f-c413-47b0-9028-c360025554fb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:19.614021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812f16f-c413-47b0-9028-c360025554fb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:19.614297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812f16f-c413-47b0-9028-c360025554fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:19.635125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812f16f-c413-47b0-9028-c360025554fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:19.636588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812f16f-c413-47b0-9028-c360025554fb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:19.638389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0890a9f-c464-4e50-8c8d-a098c39a5c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:19.640363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890a9f-c464-4e50-8c8d-a098c39a5c12 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:19.640951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890a9f-c464-4e50-8c8d-a098c39a5c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:19.641426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0890a9f-c464-4e50-8c8d-a098c39a5c12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9812f16f-c413-47b0-9028-c360025554fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:19.658598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:19.659615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:19.660578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:19.660578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:19.660578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:19.661247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 296/598] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:19.661892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0890a9f-c464-4e50-8c8d-a098c39a5c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:19.663538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0890a9f-c464-4e50-8c8d-a098c39a5c12 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:19.665442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812f16f-c413-47b0-9028-c360025554fb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a0890a9f-c464-4e50-8c8d-a098c39a5c12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:19.683120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:19.684176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:19.685042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:19.685042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:19.685042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:19.687814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 303/599] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:19.689791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb21bb2-ce74-4349-b1a4-acae0ddd3ad3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:19.690274 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb21bb2-ce74-4349-b1a4-acae0ddd3ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:19.690847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb21bb2-ce74-4349-b1a4-acae0ddd3ad3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:19.694203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 304/600] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:19.695392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb21bb2-ce74-4349-b1a4-acae0ddd3ad3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:19.785038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:19.815014 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812f16f-c413-47b0-9028-c360025554fb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:19.836278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb21bb2-ce74-4349-b1a4-acae0ddd3ad3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:19.859952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afe492-d075-4d8d-b985-915c72fa0841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:19.860970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afe492-d075-4d8d-b985-915c72fa0841 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:19.861480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afe492-d075-4d8d-b985-915c72fa0841 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:19.861986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afe492-d075-4d8d-b985-915c72fa0841 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:19.863441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afe492-d075-4d8d-b985-915c72fa0841 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:19.864222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afe492-d075-4d8d-b985-915c72fa0841 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:19.866589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afe492-d075-4d8d-b985-915c72fa0841 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:19.871285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:19.879789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51373-abf4-428e-8cc1-d4961f6ef8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:19.880505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51373-abf4-428e-8cc1-d4961f6ef8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:19.880863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51373-abf4-428e-8cc1-d4961f6ef8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:19.881217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e51373-abf4-428e-8cc1-d4961f6ef8b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:19.882417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e51373-abf4-428e-8cc1-d4961f6ef8b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:19.882822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e51373-abf4-428e-8cc1-d4961f6ef8b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:19.884641 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e51373-abf4-428e-8cc1-d4961f6ef8b5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:19.886613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 297/601] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:19.889421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f82f43-798f-4fc1-b35b-7a7a1a13ae44 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:19.890136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f82f43-798f-4fc1-b35b-7a7a1a13ae44 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:19.890588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f82f43-798f-4fc1-b35b-7a7a1a13ae44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:19.906348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 305/602] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:19.907928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51373-abf4-428e-8cc1-d4961f6ef8b5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:19.914068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-41f82f43-798f-4fc1-b35b-7a7a1a13ae44 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:19.915561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-41f82f43-798f-4fc1-b35b-7a7a1a13ae44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:20.002723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 298/603] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:20.003606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f82f43-798f-4fc1-b35b-7a7a1a13ae44 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:20.075188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e51373-abf4-428e-8cc1-d4961f6ef8b5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:20.134164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:20.150443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd118ce2-48ae-4691-a292-62e668e4b08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:20.151687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd118ce2-48ae-4691-a292-62e668e4b08d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:20.151980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd118ce2-48ae-4691-a292-62e668e4b08d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:20.152784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd118ce2-48ae-4691-a292-62e668e4b08d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:20.163767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd118ce2-48ae-4691-a292-62e668e4b08d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:20.165526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd118ce2-48ae-4691-a292-62e668e4b08d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:20.176356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f82f43-798f-4fc1-b35b-7a7a1a13ae44 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dd118ce2-48ae-4691-a292-62e668e4b08d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:20.182093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:20.182972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:20.183831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:20.183831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:20.183831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:20.183831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:20.183831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:20.183831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:20.183831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:20.183831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:20.183831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 306/604] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:20.190758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2300ba-52be-4c8a-8ead-01019d2bb373 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:20.191047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2300ba-52be-4c8a-8ead-01019d2bb373 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:20.191337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2300ba-52be-4c8a-8ead-01019d2bb373 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:20.193806 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 307/605] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:20 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:20.194642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2300ba-52be-4c8a-8ead-01019d2bb373 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:20.214258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:20.231121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77da86a-0bba-4455-9881-0d324280705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:20.232945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77da86a-0bba-4455-9881-0d324280705f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:20.233416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77da86a-0bba-4455-9881-0d324280705f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:20.234171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77da86a-0bba-4455-9881-0d324280705f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:20.243607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77da86a-0bba-4455-9881-0d324280705f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:20.246151 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77da86a-0bba-4455-9881-0d324280705f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e77da86a-0bba-4455-9881-0d324280705f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:20.263042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:20.264534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:20.265413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:20.265413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:20.265413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:20.265413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:20.265413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:20.265413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:20.265413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:20.265413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:20.272208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 299/606] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:20.272208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f7db8-a3c7-479a-a083-503e7c7c6765 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:20.272208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f7db8-a3c7-479a-a083-503e7c7c6765 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:20.272208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079f7db8-a3c7-479a-a083-503e7c7c6765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:20.294972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-079f7db8-a3c7-479a-a083-503e7c7c6765 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:20.297240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-079f7db8-a3c7-479a-a083-503e7c7c6765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:20.320079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2300ba-52be-4c8a-8ead-01019d2bb373 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:20.364237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49165ea-fb87-40fb-b5c3-2d328f080d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:20.365531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49165ea-fb87-40fb-b5c3-2d328f080d29 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:20.366095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49165ea-fb87-40fb-b5c3-2d328f080d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:20.366552 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49165ea-fb87-40fb-b5c3-2d328f080d29 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:20.367433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49165ea-fb87-40fb-b5c3-2d328f080d29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:20.368489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49165ea-fb87-40fb-b5c3-2d328f080d29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:20.371036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49165ea-fb87-40fb-b5c3-2d328f080d29 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:20.385936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 300/607] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:20.386962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f7db8-a3c7-479a-a083-503e7c7c6765 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:20.392206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 308/608] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:20.393296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49165ea-fb87-40fb-b5c3-2d328f080d29 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:20.453593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:20.527465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49165ea-fb87-40fb-b5c3-2d328f080d29 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:20.566931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079f7db8-a3c7-479a-a083-503e7c7c6765 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:20.572491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec385497-549e-42b9-99b1-d37b17b61252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:20.573381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec385497-549e-42b9-99b1-d37b17b61252 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:20.573886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec385497-549e-42b9-99b1-d37b17b61252 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:20.574362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec385497-549e-42b9-99b1-d37b17b61252 None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/b957b367ddf2483f84056ccec57fc515/roles/a797bb15d18b42148f36a77bd6d59606` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:20.593069 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec385497-549e-42b9-99b1-d37b17b61252 None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=b957b367ddf2483f84056ccec57fc515, role_id=a797bb15d18b42148f36a77bd6d59606)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:20.595171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec385497-549e-42b9-99b1-d37b17b61252 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:20.596940 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:20.622122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 309/609] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:55:20 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/b957b367ddf2483f84056ccec57fc515/roles/a797bb15d18b42148f36a77bd6d59606 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:55:20.622122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec385497-549e-42b9-99b1-d37b17b61252 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:20.640716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab978787-f3f8-4059-9b04-237d732d9b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:20.641361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab978787-f3f8-4059-9b04-237d732d9b83 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:20.641639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab978787-f3f8-4059-9b04-237d732d9b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:20.641950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab978787-f3f8-4059-9b04-237d732d9b83 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:20.642666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab978787-f3f8-4059-9b04-237d732d9b83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:20.642970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab978787-f3f8-4059-9b04-237d732d9b83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:20.645258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab978787-f3f8-4059-9b04-237d732d9b83 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:20.661145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 301/610] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:20.669400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab978787-f3f8-4059-9b04-237d732d9b83 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:20.762753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:20.771406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec385497-549e-42b9-99b1-d37b17b61252 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:20.811821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2395290-0a17-446f-b751-96d75bc0b4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:20.812741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2395290-0a17-446f-b751-96d75bc0b4be None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:20.813077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2395290-0a17-446f-b751-96d75bc0b4be None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:20.813488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2395290-0a17-446f-b751-96d75bc0b4be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:20.828740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2395290-0a17-446f-b751-96d75bc0b4be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e2395290-0a17-446f-b751-96d75bc0b4be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:20.833710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:20.834986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 310/611] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:20.836359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2395290-0a17-446f-b751-96d75bc0b4be None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:20.843874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab978787-f3f8-4059-9b04-237d732d9b83 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:20.890176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb9bf6-0a7c-4cb9-ab3b-e98bdbfe1e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:20.891021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb9bf6-0a7c-4cb9-ab3b-e98bdbfe1e1a None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:20.891306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb9bf6-0a7c-4cb9-ab3b-e98bdbfe1e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:20.891579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb9bf6-0a7c-4cb9-ab3b-e98bdbfe1e1a None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/b37a4275e7c6441aa9e8bb0298f56589/roles/a797bb15d18b42148f36a77bd6d59606` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:20.909008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb9bf6-0a7c-4cb9-ab3b-e98bdbfe1e1a None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=b37a4275e7c6441aa9e8bb0298f56589, role_id=a797bb15d18b42148f36a77bd6d59606)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:20.910494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb9bf6-0a7c-4cb9-ab3b-e98bdbfe1e1a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:20.932674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 302/612] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:55:20 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/b37a4275e7c6441aa9e8bb0298f56589/roles/a797bb15d18b42148f36a77bd6d59606 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:55:20.933538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb9bf6-0a7c-4cb9-ab3b-e98bdbfe1e1a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:20.963204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2395290-0a17-446f-b751-96d75bc0b4be None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:20.972354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:21.024309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53de36-cc7f-42cb-96c1-0e03872f80ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:21.024965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53de36-cc7f-42cb-96c1-0e03872f80ae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:21.025239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53de36-cc7f-42cb-96c1-0e03872f80ae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:21.025507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53de36-cc7f-42cb-96c1-0e03872f80ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:21.065801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53de36-cc7f-42cb-96c1-0e03872f80ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-cf53de36-cc7f-42cb-96c1-0e03872f80ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:21.067726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:21.068647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:21.069485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:21.077808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 311/613] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:21.083689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53de36-cc7f-42cb-96c1-0e03872f80ae None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:21.086015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb9bf6-0a7c-4cb9-ab3b-e98bdbfe1e1a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:21.128977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:21.169435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdef55-941c-49db-9f91-42cda4bee6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:21.175111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdef55-941c-49db-9f91-42cda4bee6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:21.175465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdef55-941c-49db-9f91-42cda4bee6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:21.175846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cdef55-941c-49db-9f91-42cda4bee6b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:21.176584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdef55-941c-49db-9f91-42cda4bee6b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:21.176889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdef55-941c-49db-9f91-42cda4bee6b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:21.178412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cdef55-941c-49db-9f91-42cda4bee6b5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:21.196876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 303/614] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:21.198271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdef55-941c-49db-9f91-42cda4bee6b5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:21.257512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53de36-cc7f-42cb-96c1-0e03872f80ae None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:21.299288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b6033-7444-4ed7-9ab2-f34d42d4aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:21.299903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b6033-7444-4ed7-9ab2-f34d42d4aef3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:21.300184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b6033-7444-4ed7-9ab2-f34d42d4aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:21.300497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b6033-7444-4ed7-9ab2-f34d42d4aef3 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:21.301171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b6033-7444-4ed7-9ab2-f34d42d4aef3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:21.301496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b6033-7444-4ed7-9ab2-f34d42d4aef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:21.302999 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b6033-7444-4ed7-9ab2-f34d42d4aef3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:21.308120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:21.313423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 312/615] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:21.318945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b6033-7444-4ed7-9ab2-f34d42d4aef3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:21.322570 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cdef55-941c-49db-9f91-42cda4bee6b5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:21.360290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538d62e-01d2-4dee-882c-166d7b271a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:21.360921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538d62e-01d2-4dee-882c-166d7b271a70 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:21.361360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538d62e-01d2-4dee-882c-166d7b271a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:21.361634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538d62e-01d2-4dee-882c-166d7b271a70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:21.392107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538d62e-01d2-4dee-882c-166d7b271a70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:21.393619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538d62e-01d2-4dee-882c-166d7b271a70 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7538d62e-01d2-4dee-882c-166d7b271a70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:21.429585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:21.431861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:21.434057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:21.434057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:21.434057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:21.436894 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 304/616] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:21.453350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:21.454532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538d62e-01d2-4dee-882c-166d7b271a70 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:21.536955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b6033-7444-4ed7-9ab2-f34d42d4aef3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:21.574558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538d62e-01d2-4dee-882c-166d7b271a70 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:21.575106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7106f8-444b-41cc-9068-adf783efc1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:21.575639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7106f8-444b-41cc-9068-adf783efc1bf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:21.575917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7106f8-444b-41cc-9068-adf783efc1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:21.576976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7106f8-444b-41cc-9068-adf783efc1bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:21.591357 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7106f8-444b-41cc-9068-adf783efc1bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:21.593203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7106f8-444b-41cc-9068-adf783efc1bf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:21.611711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd533c-fd78-4f2c-96e5-08cf2775fe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4a7106f8-444b-41cc-9068-adf783efc1bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:21.612097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:21.613009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:21.613938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:21.613938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:21.613938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:21.614106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd533c-fd78-4f2c-96e5-08cf2775fe4c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:21.614106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd533c-fd78-4f2c-96e5-08cf2775fe4c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:21.614106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd533c-fd78-4f2c-96e5-08cf2775fe4c None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:21.614106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd533c-fd78-4f2c-96e5-08cf2775fe4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:21.614954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 313/617] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:21.615094 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd533c-fd78-4f2c-96e5-08cf2775fe4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:21.615235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3d5a7-96bc-4cf8-a0bf-3e47de979851 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:21.615235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3d5a7-96bc-4cf8-a0bf-3e47de979851 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:21.615502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd533c-fd78-4f2c-96e5-08cf2775fe4c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:21.615659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f3d5a7-96bc-4cf8-a0bf-3e47de979851 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:21.617785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 314/618] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:21.619346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3d5a7-96bc-4cf8-a0bf-3e47de979851 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:21.630150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:21.635588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 305/619] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:21.636857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ce93b-c2a7-40c8-8402-3a94de7883d4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:21.637199 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ce93b-c2a7-40c8-8402-3a94de7883d4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:21.637482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ce93b-c2a7-40c8-8402-3a94de7883d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:21.657619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4a4ce93b-c2a7-40c8-8402-3a94de7883d4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:21.658924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4a4ce93b-c2a7-40c8-8402-3a94de7883d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:21.750290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 306/620] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:21.751179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ce93b-c2a7-40c8-8402-3a94de7883d4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:21.771636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f3d5a7-96bc-4cf8-a0bf-3e47de979851 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:21.818705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:21.825063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c3175-07f5-4ab3-bd5e-e6d7078d93f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:21.825686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c3175-07f5-4ab3-bd5e-e6d7078d93f0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:21.826054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c3175-07f5-4ab3-bd5e-e6d7078d93f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:21.826476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c3175-07f5-4ab3-bd5e-e6d7078d93f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:21.827422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c3175-07f5-4ab3-bd5e-e6d7078d93f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:21.827726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c3175-07f5-4ab3-bd5e-e6d7078d93f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:21.829107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c3175-07f5-4ab3-bd5e-e6d7078d93f0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:21.843007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 315/621] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:21.848478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c3175-07f5-4ab3-bd5e-e6d7078d93f0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:21.903533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ce93b-c2a7-40c8-8402-3a94de7883d4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:21.941940 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:21.958672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9b181-0627-4aa1-9d9e-c25b06a0ab3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:21.959691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9b181-0627-4aa1-9d9e-c25b06a0ab3a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:21.960177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9b181-0627-4aa1-9d9e-c25b06a0ab3a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:21.960655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9b181-0627-4aa1-9d9e-c25b06a0ab3a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:21.971334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c3175-07f5-4ab3-bd5e-e6d7078d93f0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:21.974107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9b181-0627-4aa1-9d9e-c25b06a0ab3a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:21.981787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9b181-0627-4aa1-9d9e-c25b06a0ab3a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-22a9b181-0627-4aa1-9d9e-c25b06a0ab3a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:21.995718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:21.996717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:21.997671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 307/622] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Aug 19 20:55:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:22.003771 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9b181-0627-4aa1-9d9e-c25b06a0ab3a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:22.011373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8f0b5-a4d9-4783-8156-fbd309e61551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:22.012006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8f0b5-a4d9-4783-8156-fbd309e61551 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:22.012279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8f0b5-a4d9-4783-8156-fbd309e61551 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:22.012549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8f0b5-a4d9-4783-8156-fbd309e61551 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:22.020698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8f0b5-a4d9-4783-8156-fbd309e61551 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:22.022223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8f0b5-a4d9-4783-8156-fbd309e61551 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f7a8f0b5-a4d9-4783-8156-fbd309e61551 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:22.055693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:22.057979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:22.060211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 316/623] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:22.063388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8f0b5-a4d9-4783-8156-fbd309e61551 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:22.126168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:22.147123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9b181-0627-4aa1-9d9e-c25b06a0ab3a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:22.180240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8f0b5-a4d9-4783-8156-fbd309e61551 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:22.193408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15854b9f-2c79-418e-90b3-3262b42b2e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:22.194621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15854b9f-2c79-418e-90b3-3262b42b2e76 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:22.195112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15854b9f-2c79-418e-90b3-3262b42b2e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:22.195562 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15854b9f-2c79-418e-90b3-3262b42b2e76 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:22.196486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15854b9f-2c79-418e-90b3-3262b42b2e76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:22.196970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15854b9f-2c79-418e-90b3-3262b42b2e76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:22.199564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15854b9f-2c79-418e-90b3-3262b42b2e76 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:22.222620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c9a16-f74e-4e59-96e4-9e3e314a9ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:22.223339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c9a16-f74e-4e59-96e4-9e3e314a9ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:22.223679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c9a16-f74e-4e59-96e4-9e3e314a9ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:22.224022 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c9a16-f74e-4e59-96e4-9e3e314a9ec1 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:22.224819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c9a16-f74e-4e59-96e4-9e3e314a9ec1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:22.229510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c9a16-f74e-4e59-96e4-9e3e314a9ec1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:22.229510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c9a16-f74e-4e59-96e4-9e3e314a9ec1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:22.230146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 308/624] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Aug 19 20:55:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:22.231734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15854b9f-2c79-418e-90b3-3262b42b2e76 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:22.259165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 317/625] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Aug 19 20:55:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:22.259165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c9a16-f74e-4e59-96e4-9e3e314a9ec1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:22.291614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:22.356411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15854b9f-2c79-418e-90b3-3262b42b2e76 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:22.398591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c9a16-f74e-4e59-96e4-9e3e314a9ec1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:22.406433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d410b4-0bf3-44cd-93aa-bff291519e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:22.407491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d410b4-0bf3-44cd-93aa-bff291519e57 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:22.408663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d410b4-0bf3-44cd-93aa-bff291519e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:22.409173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d410b4-0bf3-44cd-93aa-bff291519e57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:22.410972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:22.428138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d410b4-0bf3-44cd-93aa-bff291519e57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-78d410b4-0bf3-44cd-93aa-bff291519e57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:22.436814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:22.437809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:22.438638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:22.439252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 309/626] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:22.439943 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d410b4-0bf3-44cd-93aa-bff291519e57 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:22.443572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2b2d1-9e2f-48e4-b08c-156bb17e03b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:22.444259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2b2d1-9e2f-48e4-b08c-156bb17e03b8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:22.444454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2b2d1-9e2f-48e4-b08c-156bb17e03b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:22.444719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2b2d1-9e2f-48e4-b08c-156bb17e03b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:22.526203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2b2d1-9e2f-48e4-b08c-156bb17e03b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:22.527691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2b2d1-9e2f-48e4-b08c-156bb17e03b8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:22.563251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d410b4-0bf3-44cd-93aa-bff291519e57 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:22.573499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:22.602421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9e48f-6e90-49c5-b831-f3dbac4d5340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:22.603836 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9e48f-6e90-49c5-b831-f3dbac4d5340 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:22.604356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9e48f-6e90-49c5-b831-f3dbac4d5340 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:22.604828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9e48f-6e90-49c5-b831-f3dbac4d5340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:22.616937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9e48f-6e90-49c5-b831-f3dbac4d5340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:22.618991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9e48f-6e90-49c5-b831-f3dbac4d5340 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5bf9e48f-6e90-49c5-b831-f3dbac4d5340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:22.635717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:22.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:22.637643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:22.637643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:22.637643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:22.641811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 318/628] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:22.642280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2b2d1-9e2f-48e4-b08c-156bb17e03b8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:22.644400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 310/628] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:22.645502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9e48f-6e90-49c5-b831-f3dbac4d5340 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:22.653329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:22.749790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2b2d1-9e2f-48e4-b08c-156bb17e03b8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:22.761564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9e48f-6e90-49c5-b831-f3dbac4d5340 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:22.786168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad6940-5f6d-4652-a5cb-a3407476096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:22.786851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad6940-5f6d-4652-a5cb-a3407476096d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:22.787208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad6940-5f6d-4652-a5cb-a3407476096d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:22.787552 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad6940-5f6d-4652-a5cb-a3407476096d None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:22.788337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad6940-5f6d-4652-a5cb-a3407476096d None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:22.788748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad6940-5f6d-4652-a5cb-a3407476096d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:22.790261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad6940-5f6d-4652-a5cb-a3407476096d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:22.800324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 319/629] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:22.800881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdacb5d-7168-4500-bc76-e2e45eda1087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:22.801416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad6940-5f6d-4652-a5cb-a3407476096d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:22.801577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdacb5d-7168-4500-bc76-e2e45eda1087 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:22.802037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdacb5d-7168-4500-bc76-e2e45eda1087 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:22.802583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdacb5d-7168-4500-bc76-e2e45eda1087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:22.816215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdacb5d-7168-4500-bc76-e2e45eda1087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:22.817695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdacb5d-7168-4500-bc76-e2e45eda1087 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4cdacb5d-7168-4500-bc76-e2e45eda1087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:22.834343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:22.836219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:22.838388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:22.838388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:22.838388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:22.844439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 311/630] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:22.844439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdacb5d-7168-4500-bc76-e2e45eda1087 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:22.864629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:22.887561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:22.927969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad6940-5f6d-4652-a5cb-a3407476096d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:22.964919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf0461-3492-4d09-bf4f-2c70ecea2980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:22.965556 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf0461-3492-4d09-bf4f-2c70ecea2980 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:22.965860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf0461-3492-4d09-bf4f-2c70ecea2980 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:22.966138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf0461-3492-4d09-bf4f-2c70ecea2980 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:22.966858 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf0461-3492-4d09-bf4f-2c70ecea2980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:22.967397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf0461-3492-4d09-bf4f-2c70ecea2980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:22.969009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf0461-3492-4d09-bf4f-2c70ecea2980 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:22.973423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdacb5d-7168-4500-bc76-e2e45eda1087 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:22.987973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 320/631] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:22.988768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf0461-3492-4d09-bf4f-2c70ecea2980 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:23.014100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8df79-57f6-431a-a022-ef4dc403d828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:23.015107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8df79-57f6-431a-a022-ef4dc403d828 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.015402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8df79-57f6-431a-a022-ef4dc403d828 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.015666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8df79-57f6-431a-a022-ef4dc403d828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.026701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8df79-57f6-431a-a022-ef4dc403d828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:23.028165 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8df79-57f6-431a-a022-ef4dc403d828 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-98d8df79-57f6-431a-a022-ef4dc403d828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:23.044295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:23.045380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:23.046752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:23.046752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:23.046752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:23.046752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 312/632] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:23.047342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478a599-4f98-45cd-a3dd-cf31a2bfa17f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.047635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478a599-4f98-45cd-a3dd-cf31a2bfa17f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.047920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4478a599-4f98-45cd-a3dd-cf31a2bfa17f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.053927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 313/633] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:23.053927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478a599-4f98-45cd-a3dd-cf31a2bfa17f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:23.113951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf0461-3492-4d09-bf4f-2c70ecea2980 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:23.133511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:23.147259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:23.152843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a20235-ce96-4957-b609-61f91d8ce1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:23.153651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a20235-ce96-4957-b609-61f91d8ce1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.154749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a20235-ce96-4957-b609-61f91d8ce1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.155659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a20235-ce96-4957-b609-61f91d8ce1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.155836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a20235-ce96-4957-b609-61f91d8ce1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:23.156088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a20235-ce96-4957-b609-61f91d8ce1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:23.157496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a20235-ce96-4957-b609-61f91d8ce1a5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:23.174279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 321/634] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:23.175774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a20235-ce96-4957-b609-61f91d8ce1a5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:23.187898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4478a599-4f98-45cd-a3dd-cf31a2bfa17f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:23.224111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae5a40-4504-4fe4-9790-4b42ea06c19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:23.225069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae5a40-4504-4fe4-9790-4b42ea06c19b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.225530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae5a40-4504-4fe4-9790-4b42ea06c19b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.226036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae5a40-4504-4fe4-9790-4b42ea06c19b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.233088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae5a40-4504-4fe4-9790-4b42ea06c19b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:23.234588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae5a40-4504-4fe4-9790-4b42ea06c19b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-caae5a40-4504-4fe4-9790-4b42ea06c19b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:23.246235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:23.247348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:23.248797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:23.248797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:23.248797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:23.248797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:23.248797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:23.248797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:23.248797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:23.248797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:23.248797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 314/635] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:23.249432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1dff5-0f51-42fc-8c1f-c8720c77fb75 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.249737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1dff5-0f51-42fc-8c1f-c8720c77fb75 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.250017 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe1dff5-0f51-42fc-8c1f-c8720c77fb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.269390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dbe1dff5-0f51-42fc-8c1f-c8720c77fb75 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:23.270620 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dbe1dff5-0f51-42fc-8c1f-c8720c77fb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:23.293877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a20235-ce96-4957-b609-61f91d8ce1a5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:23.330535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f8577-1298-4057-8a55-245dbac989e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:23.331057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f8577-1298-4057-8a55-245dbac989e8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.331334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f8577-1298-4057-8a55-245dbac989e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.331600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f8577-1298-4057-8a55-245dbac989e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.332231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f8577-1298-4057-8a55-245dbac989e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:23.332528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f8577-1298-4057-8a55-245dbac989e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:23.333940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f8577-1298-4057-8a55-245dbac989e8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:23.339834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 315/636] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:23.340679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1dff5-0f51-42fc-8c1f-c8720c77fb75 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:23.349967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 322/637] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:23.349967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f8577-1298-4057-8a55-245dbac989e8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:23.405002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:23.407867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:23.455772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe1dff5-0f51-42fc-8c1f-c8720c77fb75 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:23.465886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f8577-1298-4057-8a55-245dbac989e8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:23.497591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560109a-9bc7-4bcd-a2eb-bde197a8648c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:23.499953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560109a-9bc7-4bcd-a2eb-bde197a8648c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.500448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560109a-9bc7-4bcd-a2eb-bde197a8648c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.500960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560109a-9bc7-4bcd-a2eb-bde197a8648c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.508173 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2df40e-f710-4a79-aec3-721fcbc2cae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:23.508780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2df40e-f710-4a79-aec3-721fcbc2cae4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.509057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2df40e-f710-4a79-aec3-721fcbc2cae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.509323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2df40e-f710-4a79-aec3-721fcbc2cae4 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.510041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2df40e-f710-4a79-aec3-721fcbc2cae4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:23.510346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2df40e-f710-4a79-aec3-721fcbc2cae4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:23.511778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2df40e-f710-4a79-aec3-721fcbc2cae4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:23.512884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560109a-9bc7-4bcd-a2eb-bde197a8648c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:23.517225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560109a-9bc7-4bcd-a2eb-bde197a8648c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:23.529684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 323/638] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:23.529684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc46de-6bcd-434a-ba29-a5c203d41894 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.529684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc46de-6bcd-434a-ba29-a5c203d41894 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.529684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc46de-6bcd-434a-ba29-a5c203d41894 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.529684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 324/639] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:23.529684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc46de-6bcd-434a-ba29-a5c203d41894 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c560109a-9bc7-4bcd-a2eb-bde197a8648c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:23.537128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:23.539487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:23.541258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:23.541258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:23.541258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:23.541258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:23.541258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:23.541258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:23.541258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:23.541258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:23.541258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 316/640] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:23.541258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560109a-9bc7-4bcd-a2eb-bde197a8648c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:23.658672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560109a-9bc7-4bcd-a2eb-bde197a8648c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:23.664244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc46de-6bcd-434a-ba29-a5c203d41894 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:23.672627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:23.675782 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:23.699384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e594b7cc-4a81-472a-b250-0156a5ee421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:23.700017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594b7cc-4a81-472a-b250-0156a5ee421d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.700295 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594b7cc-4a81-472a-b250-0156a5ee421d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.700571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e594b7cc-4a81-472a-b250-0156a5ee421d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.702616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4280fe-fb86-433e-a87a-b7b9c0fde53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:23.703772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4280fe-fb86-433e-a87a-b7b9c0fde53d None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.704273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4280fe-fb86-433e-a87a-b7b9c0fde53d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.704759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4280fe-fb86-433e-a87a-b7b9c0fde53d None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/61a0906b8b134c178b4dece2891f1596/roles/a797bb15d18b42148f36a77bd6d59606` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.708892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594b7cc-4a81-472a-b250-0156a5ee421d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:23.710383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e594b7cc-4a81-472a-b250-0156a5ee421d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e594b7cc-4a81-472a-b250-0156a5ee421d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:23.722046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:23.723120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:23.724074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 325/641] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:23.724657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970818f9-c975-4602-b5d7-6ccf54ef49ae None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.724657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970818f9-c975-4602-b5d7-6ccf54ef49ae None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.725302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970818f9-c975-4602-b5d7-6ccf54ef49ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.732023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4280fe-fb86-433e-a87a-b7b9c0fde53d None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=61a0906b8b134c178b4dece2891f1596, role_id=a797bb15d18b42148f36a77bd6d59606)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:23.734194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4280fe-fb86-433e-a87a-b7b9c0fde53d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:23.743374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-970818f9-c975-4602-b5d7-6ccf54ef49ae None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:23.744839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-970818f9-c975-4602-b5d7-6ccf54ef49ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:23.747645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 317/642] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:55:23 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/61a0906b8b134c178b4dece2891f1596/roles/a797bb15d18b42148f36a77bd6d59606 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:55:23.749205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4280fe-fb86-433e-a87a-b7b9c0fde53d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:23.857797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 326/643] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:23.857797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970818f9-c975-4602-b5d7-6ccf54ef49ae None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:23.906159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4280fe-fb86-433e-a87a-b7b9c0fde53d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:23.955785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517469ec-8fc3-4fe5-a590-15c6b9c2fa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:23.957360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517469ec-8fc3-4fe5-a590-15c6b9c2fa79 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:23.957676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517469ec-8fc3-4fe5-a590-15c6b9c2fa79 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:23.958548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517469ec-8fc3-4fe5-a590-15c6b9c2fa79 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:23.959426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517469ec-8fc3-4fe5-a590-15c6b9c2fa79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:23.959911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517469ec-8fc3-4fe5-a590-15c6b9c2fa79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:23.962354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517469ec-8fc3-4fe5-a590-15c6b9c2fa79 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:23.979094 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 318/644] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:23.980330 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517469ec-8fc3-4fe5-a590-15c6b9c2fa79 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:24.000194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:24.003926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970818f9-c975-4602-b5d7-6ccf54ef49ae None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:24.021102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:24.042521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333818b5-664e-41b9-97b4-bb36267533c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:24.043324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333818b5-664e-41b9-97b4-bb36267533c2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:24.043707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333818b5-664e-41b9-97b4-bb36267533c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:24.044084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333818b5-664e-41b9-97b4-bb36267533c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:24.057062 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333818b5-664e-41b9-97b4-bb36267533c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-333818b5-664e-41b9-97b4-bb36267533c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:24.062831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:24.063767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:24.064735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 327/645] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:24.066667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333818b5-664e-41b9-97b4-bb36267533c2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:24.117016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517469ec-8fc3-4fe5-a590-15c6b9c2fa79 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:24.159851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be626a-6dd7-4d96-a224-d1eabdac07dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:24.160958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be626a-6dd7-4d96-a224-d1eabdac07dc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:24.161515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be626a-6dd7-4d96-a224-d1eabdac07dc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:24.161823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be626a-6dd7-4d96-a224-d1eabdac07dc None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:24.162831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be626a-6dd7-4d96-a224-d1eabdac07dc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:24.163152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be626a-6dd7-4d96-a224-d1eabdac07dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:24.165024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be626a-6dd7-4d96-a224-d1eabdac07dc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:24.180279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333818b5-664e-41b9-97b4-bb36267533c2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:24.189394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 319/646] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Aug 19 20:55:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:24.190613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be626a-6dd7-4d96-a224-d1eabdac07dc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:24.224258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167550e-184a-4fa9-8539-6d31b94427db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:24.224720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167550e-184a-4fa9-8539-6d31b94427db None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:24.225014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167550e-184a-4fa9-8539-6d31b94427db None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:24.225341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5167550e-184a-4fa9-8539-6d31b94427db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:24.226064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167550e-184a-4fa9-8539-6d31b94427db None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=43a9ad5341b54c40a5427045660363c3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:24.226562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167550e-184a-4fa9-8539-6d31b94427db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:24.227872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5167550e-184a-4fa9-8539-6d31b94427db None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:24.235027 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 328/647] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:24 2022] GET /identity/v3/role_assignments?user.id=43a9ad5341b54c40a5427045660363c3&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:24.240936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167550e-184a-4fa9-8539-6d31b94427db None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:24.276406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:24.291253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:24.315225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be626a-6dd7-4d96-a224-d1eabdac07dc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:24.356194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518c2d9-678d-450c-aad0-7d5b97a62b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:24.356828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c2d9-678d-450c-aad0-7d5b97a62b80 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:24.357107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c2d9-678d-450c-aad0-7d5b97a62b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:24.358129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518c2d9-678d-450c-aad0-7d5b97a62b80 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:24.359202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518c2d9-678d-450c-aad0-7d5b97a62b80 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:24.359391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518c2d9-678d-450c-aad0-7d5b97a62b80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:24.360784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518c2d9-678d-450c-aad0-7d5b97a62b80 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:24.369794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5167550e-184a-4fa9-8539-6d31b94427db None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:24.376421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 320/648] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:24.377443 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518c2d9-678d-450c-aad0-7d5b97a62b80 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:24.411382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564c9f3-cf2d-429f-baa1-15f46cd30d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:24.412427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564c9f3-cf2d-429f-baa1-15f46cd30d70 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:24.412902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564c9f3-cf2d-429f-baa1-15f46cd30d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:24.413547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564c9f3-cf2d-429f-baa1-15f46cd30d70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:24.427299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564c9f3-cf2d-429f-baa1-15f46cd30d70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2564c9f3-cf2d-429f-baa1-15f46cd30d70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:24.433128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:24.434100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:24.434929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:24.438537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 329/649] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:24.439835 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564c9f3-cf2d-429f-baa1-15f46cd30d70 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:24.512937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518c2d9-678d-450c-aad0-7d5b97a62b80 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:24.567452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4185c8-40d8-4e85-a1a4-d66598bc2716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:24.568361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4185c8-40d8-4e85-a1a4-d66598bc2716 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:24.568640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4185c8-40d8-4e85-a1a4-d66598bc2716 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:24.568913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4185c8-40d8-4e85-a1a4-d66598bc2716 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:24.575752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:24.590761 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:24.600972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564c9f3-cf2d-429f-baa1-15f46cd30d70 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:24.652509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41263a-bfe4-472b-96ea-80aa8ca1f414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:24.653795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41263a-bfe4-472b-96ea-80aa8ca1f414 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:24.653795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41263a-bfe4-472b-96ea-80aa8ca1f414 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:24.653991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea41263a-bfe4-472b-96ea-80aa8ca1f414 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:24.667623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41263a-bfe4-472b-96ea-80aa8ca1f414 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:24.668873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4185c8-40d8-4e85-a1a4-d66598bc2716 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:24.669246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea41263a-bfe4-472b-96ea-80aa8ca1f414 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:24.671128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4185c8-40d8-4e85-a1a4-d66598bc2716 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ea41263a-bfe4-472b-96ea-80aa8ca1f414 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:24.686306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:24.687299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:24.688187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:24.688187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:24.688187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:24.689065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 330/650] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:24.689205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41263a-bfe4-472b-96ea-80aa8ca1f414 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:24.783560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 321/651] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:24.784377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4185c8-40d8-4e85-a1a4-d66598bc2716 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:24.808829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea41263a-bfe4-472b-96ea-80aa8ca1f414 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:24.840671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:24.851669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f4150-225e-462a-b97d-768230df4cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:24.852285 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f4150-225e-462a-b97d-768230df4cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:24.852560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f4150-225e-462a-b97d-768230df4cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:24.852829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f4150-225e-462a-b97d-768230df4cf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:24.853494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f4150-225e-462a-b97d-768230df4cf4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:24.853815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f4150-225e-462a-b97d-768230df4cf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:24.855291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f4150-225e-462a-b97d-768230df4cf4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:24.861441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:24.877810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 331/652] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:24.877810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f4150-225e-462a-b97d-768230df4cf4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:24.938865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4185c8-40d8-4e85-a1a4-d66598bc2716 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:24.988586 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ce5f8-08ca-430c-be23-0193114b1e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:24.989355 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ce5f8-08ca-430c-be23-0193114b1e21 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:24.989740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ce5f8-08ca-430c-be23-0193114b1e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:24.990409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ce5f8-08ca-430c-be23-0193114b1e21 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:24.991175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ce5f8-08ca-430c-be23-0193114b1e21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:24.991596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ce5f8-08ca-430c-be23-0193114b1e21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:24.994309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ce5f8-08ca-430c-be23-0193114b1e21 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:25.004236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f4150-225e-462a-b97d-768230df4cf4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:25.008580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 322/653] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:25.009745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ce5f8-08ca-430c-be23-0193114b1e21 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:25.050884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82e319-acef-4f4f-a231-5e9b5dfd8667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:25.051775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82e319-acef-4f4f-a231-5e9b5dfd8667 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:25.052157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82e319-acef-4f4f-a231-5e9b5dfd8667 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:25.052540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d82e319-acef-4f4f-a231-5e9b5dfd8667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:25.067522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82e319-acef-4f4f-a231-5e9b5dfd8667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:25.069168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d82e319-acef-4f4f-a231-5e9b5dfd8667 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8d82e319-acef-4f4f-a231-5e9b5dfd8667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:25.089079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:25.090071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:25.091038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:25.091038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:25.091038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:25.093843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 332/654] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:25.094911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82e319-acef-4f4f-a231-5e9b5dfd8667 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:25.136759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ce5f8-08ca-430c-be23-0193114b1e21 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:25.139804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:25.176675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:25.183867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d52310-447b-4ba8-a74e-7b7176ba5449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:25.184834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d52310-447b-4ba8-a74e-7b7176ba5449 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:25.185357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d52310-447b-4ba8-a74e-7b7176ba5449 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:25.186546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d52310-447b-4ba8-a74e-7b7176ba5449 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:25.198580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d52310-447b-4ba8-a74e-7b7176ba5449 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:25.200336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d52310-447b-4ba8-a74e-7b7176ba5449 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-54d52310-447b-4ba8-a74e-7b7176ba5449 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:25.219829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:25.220820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:25.221701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:25.221701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:25.221701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:25.226176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 323/655] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:25.227064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d52310-447b-4ba8-a74e-7b7176ba5449 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:25.239953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d82e319-acef-4f4f-a231-5e9b5dfd8667 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:25.280183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3e0b-b7cd-4bee-8e6e-c3ec7842adc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:25.281501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3e0b-b7cd-4bee-8e6e-c3ec7842adc9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:25.282037 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3e0b-b7cd-4bee-8e6e-c3ec7842adc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:25.282549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e3e0b-b7cd-4bee-8e6e-c3ec7842adc9 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:25.291438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e3e0b-b7cd-4bee-8e6e-c3ec7842adc9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:25.293227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e3e0b-b7cd-4bee-8e6e-c3ec7842adc9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3f8e3e0b-b7cd-4bee-8e6e-c3ec7842adc9 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:25.306446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:25.307388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:25.308324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:25.308324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:25.308324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:25.308324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:25.308324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:25.308324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:25.308324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:25.308324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:25.308324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 19 20:55:25.308324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:25.311322 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 333/656] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:25.312533 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3e0b-b7cd-4bee-8e6e-c3ec7842adc9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:25.359785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d52310-447b-4ba8-a74e-7b7176ba5449 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:25.408188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2207d-8a89-41d7-a599-773df9c6558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:25.409076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2207d-8a89-41d7-a599-773df9c6558c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:25.409365 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2207d-8a89-41d7-a599-773df9c6558c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:25.409645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2207d-8a89-41d7-a599-773df9c6558c None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:25.410342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2207d-8a89-41d7-a599-773df9c6558c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:25.411051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2207d-8a89-41d7-a599-773df9c6558c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:25.412449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2207d-8a89-41d7-a599-773df9c6558c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:25.427346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e3e0b-b7cd-4bee-8e6e-c3ec7842adc9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:25.433748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 324/657] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:25.435041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2207d-8a89-41d7-a599-773df9c6558c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:25.445275 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:25.466161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:25.478418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e937d8-f892-4b03-8e6b-cdde26fcada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:25.479041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e937d8-f892-4b03-8e6b-cdde26fcada6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:25.479323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e937d8-f892-4b03-8e6b-cdde26fcada6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:25.479599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e937d8-f892-4b03-8e6b-cdde26fcada6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:25.480252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e937d8-f892-4b03-8e6b-cdde26fcada6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:25.480567 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e937d8-f892-4b03-8e6b-cdde26fcada6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:25.481980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e937d8-f892-4b03-8e6b-cdde26fcada6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:25.502075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 334/658] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:25.503119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e937d8-f892-4b03-8e6b-cdde26fcada6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:25.563185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2207d-8a89-41d7-a599-773df9c6558c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:25.603719 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8e0b7-7257-42d5-82df-b420cb9d4042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:25.604359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8e0b7-7257-42d5-82df-b420cb9d4042 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:25.604643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8e0b7-7257-42d5-82df-b420cb9d4042 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:25.604978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8e0b7-7257-42d5-82df-b420cb9d4042 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:25.605744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8e0b7-7257-42d5-82df-b420cb9d4042 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:25.606075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8e0b7-7257-42d5-82df-b420cb9d4042 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:25.608185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8e0b7-7257-42d5-82df-b420cb9d4042 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:25.628053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 325/659] 10.222.0.16 () {56 vars in 1137 bytes} [Fri Aug 19 20:55:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:25.628971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8e0b7-7257-42d5-82df-b420cb9d4042 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:25.634600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e937d8-f892-4b03-8e6b-cdde26fcada6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:25.671756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cc809-f2a0-4c95-957a-47a4eb67bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:25.672396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cc809-f2a0-4c95-957a-47a4eb67bb48 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:25.672652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cc809-f2a0-4c95-957a-47a4eb67bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:25.672937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cc809-f2a0-4c95-957a-47a4eb67bb48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:25.680227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714cc809-f2a0-4c95-957a-47a4eb67bb48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:25.681705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714cc809-f2a0-4c95-957a-47a4eb67bb48 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-714cc809-f2a0-4c95-957a-47a4eb67bb48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:25.694748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:25.695657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:25.696531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:25.696531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:25.696531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:25.696531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:25.696531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:25.696531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:25.696531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:25.696531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:25.700739 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 335/660] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:25.702645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cc809-f2a0-4c95-957a-47a4eb67bb48 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:25.733926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:25.740161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:25.770388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8e0b7-7257-42d5-82df-b420cb9d4042 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:25.814908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605ff04-7ae6-4543-81ea-508232f9bbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:25.816328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605ff04-7ae6-4543-81ea-508232f9bbbe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:25.816771 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605ff04-7ae6-4543-81ea-508232f9bbbe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:25.817189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605ff04-7ae6-4543-81ea-508232f9bbbe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:25.821517 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cc809-f2a0-4c95-957a-47a4eb67bb48 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:25.828155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605ff04-7ae6-4543-81ea-508232f9bbbe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:25.829888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605ff04-7ae6-4543-81ea-508232f9bbbe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b605ff04-7ae6-4543-81ea-508232f9bbbe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:25.841466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:25.843149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:25.844069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 326/661] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:25.844764 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605ff04-7ae6-4543-81ea-508232f9bbbe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:25.862118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38390e-1e82-4812-888b-6723be2d51a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:25.862732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38390e-1e82-4812-888b-6723be2d51a7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:25.862994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38390e-1e82-4812-888b-6723be2d51a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:25.863262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38390e-1e82-4812-888b-6723be2d51a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:25.938578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38390e-1e82-4812-888b-6723be2d51a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:25.940048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38390e-1e82-4812-888b-6723be2d51a7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:25.968704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605ff04-7ae6-4543-81ea-508232f9bbbe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:25.970706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:26.012610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb2b86-76c9-449e-890b-c7c26d76643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:26.013226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb2b86-76c9-449e-890b-c7c26d76643a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.013736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb2b86-76c9-449e-890b-c7c26d76643a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.014088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb2b86-76c9-449e-890b-c7c26d76643a None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.014751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb2b86-76c9-449e-890b-c7c26d76643a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:26.015056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb2b86-76c9-449e-890b-c7c26d76643a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:26.016523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb2b86-76c9-449e-890b-c7c26d76643a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:26.018410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:26.032982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 327/662] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:26.034307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dd48c-7508-417e-b5e2-39962a0fedf5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.034934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dd48c-7508-417e-b5e2-39962a0fedf5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.035657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3dd48c-7508-417e-b5e2-39962a0fedf5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.041795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 328/663] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:26.041795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dd48c-7508-417e-b5e2-39962a0fedf5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:26.061305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 336/664] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:26.061305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38390e-1e82-4812-888b-6723be2d51a7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:26.158890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3dd48c-7508-417e-b5e2-39962a0fedf5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:26.178111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38390e-1e82-4812-888b-6723be2d51a7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:26.200145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94507bc2-70e3-420b-bcad-1d744c96a590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:26.201575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94507bc2-70e3-420b-bcad-1d744c96a590 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.201894 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94507bc2-70e3-420b-bcad-1d744c96a590 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.202191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94507bc2-70e3-420b-bcad-1d744c96a590 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.202981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94507bc2-70e3-420b-bcad-1d744c96a590 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:26.203216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94507bc2-70e3-420b-bcad-1d744c96a590 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:26.204656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94507bc2-70e3-420b-bcad-1d744c96a590 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:26.211575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:26.220453 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83eaf0-83f2-4a51-80ec-805755fc735d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:26.221487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83eaf0-83f2-4a51-80ec-805755fc735d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.221894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83eaf0-83f2-4a51-80ec-805755fc735d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.222269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83eaf0-83f2-4a51-80ec-805755fc735d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.227635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 329/665] 10.222.0.16 () {56 vars in 1139 bytes} [Fri Aug 19 20:55:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:26.227785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83eaf0-83f2-4a51-80ec-805755fc735d None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=54d1ae400b5c4ae0955047b9e7750103) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:26.227785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83eaf0-83f2-4a51-80ec-805755fc735d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:26.227945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924de325-cc28-44f0-b984-6ae39a1d654c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.227945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924de325-cc28-44f0-b984-6ae39a1d654c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.227945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924de325-cc28-44f0-b984-6ae39a1d654c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.228183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc83eaf0-83f2-4a51-80ec-805755fc735d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:26.232399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 337/666] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:26 2022] GET /identity/v3/role_assignments?user.id=54d1ae400b5c4ae0955047b9e7750103&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:26.242056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83eaf0-83f2-4a51-80ec-805755fc735d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:26.244188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-924de325-cc28-44f0-b984-6ae39a1d654c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:26.245549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-924de325-cc28-44f0-b984-6ae39a1d654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:26.308149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:26.342282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 330/667] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:26.343436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924de325-cc28-44f0-b984-6ae39a1d654c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:26.402981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83eaf0-83f2-4a51-80ec-805755fc735d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:26.444944 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911188c-5256-4a85-97e5-4bac0441064f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:26.445587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911188c-5256-4a85-97e5-4bac0441064f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.445902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911188c-5256-4a85-97e5-4bac0441064f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.446179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911188c-5256-4a85-97e5-4bac0441064f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.463721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911188c-5256-4a85-97e5-4bac0441064f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:26.465213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911188c-5256-4a85-97e5-4bac0441064f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d911188c-5256-4a85-97e5-4bac0441064f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:26.483473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:26.484725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:26.485842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:26.485842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:26.485842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:26.485842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 338/668] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:26.486122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3bdc-fdb4-4231-8a30-dd222f93faba None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.486470 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3bdc-fdb4-4231-8a30-dd222f93faba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.486751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3bdc-fdb4-4231-8a30-dd222f93faba None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.492389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 339/669] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:26.493554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e3bdc-fdb4-4231-8a30-dd222f93faba None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:26.505019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924de325-cc28-44f0-b984-6ae39a1d654c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:26.547989 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd1790-f704-4da7-8343-da96c6a82fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:26.548902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1790-f704-4da7-8343-da96c6a82fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.549185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1790-f704-4da7-8343-da96c6a82fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.549462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfd1790-f704-4da7-8343-da96c6a82fa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.555247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:26.625167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:26.627077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e3bdc-fdb4-4231-8a30-dd222f93faba None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:26.650167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd1790-f704-4da7-8343-da96c6a82fa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:26.651636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfd1790-f704-4da7-8343-da96c6a82fa8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:26.668554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e3168-1637-4398-a7a2-66d1b1a4b979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:26.669313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e3168-1637-4398-a7a2-66d1b1a4b979 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.669733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e3168-1637-4398-a7a2-66d1b1a4b979 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.670128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e3168-1637-4398-a7a2-66d1b1a4b979 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.670960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e3168-1637-4398-a7a2-66d1b1a4b979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:26.671383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e3168-1637-4398-a7a2-66d1b1a4b979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:26.672886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e3168-1637-4398-a7a2-66d1b1a4b979 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:26.689883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 340/670] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:26.689883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16e8ba-6bb6-489c-a39e-1c84a123100f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.689883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16e8ba-6bb6-489c-a39e-1c84a123100f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.689883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16e8ba-6bb6-489c-a39e-1c84a123100f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.708160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7b16e8ba-6bb6-489c-a39e-1c84a123100f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:26.709472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7b16e8ba-6bb6-489c-a39e-1c84a123100f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:26.770819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 331/671] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:26.771839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd1790-f704-4da7-8343-da96c6a82fa8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:26.789808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 341/672] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:26.789808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16e8ba-6bb6-489c-a39e-1c84a123100f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:26.832938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:26.876009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:26.901599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfd1790-f704-4da7-8343-da96c6a82fa8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:26.925285 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16e8ba-6bb6-489c-a39e-1c84a123100f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:26.944108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb276c8-4fbb-47c3-99a4-04e422145227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:26.945029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb276c8-4fbb-47c3-99a4-04e422145227 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.945557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb276c8-4fbb-47c3-99a4-04e422145227 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.946013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb276c8-4fbb-47c3-99a4-04e422145227 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.957812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb276c8-4fbb-47c3-99a4-04e422145227 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-deb276c8-4fbb-47c3-99a4-04e422145227 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:26.963434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:26.964422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:26.965403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:26.972583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 332/673] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:26.973660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb276c8-4fbb-47c3-99a4-04e422145227 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:26.977636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47db6a-b9b9-4eb2-857f-cf94196244b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:26.978747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47db6a-b9b9-4eb2-857f-cf94196244b6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:26.979123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47db6a-b9b9-4eb2-857f-cf94196244b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:26.979498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47db6a-b9b9-4eb2-857f-cf94196244b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:26.990064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47db6a-b9b9-4eb2-857f-cf94196244b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:26.992296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47db6a-b9b9-4eb2-857f-cf94196244b6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0c47db6a-b9b9-4eb2-857f-cf94196244b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:27.007162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:27.008487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:27.009592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:27.009592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:27.009592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:27.009592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:27.009592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:27.009592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:27.009592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:27.009592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:27.011837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 342/674] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:27.012892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47db6a-b9b9-4eb2-857f-cf94196244b6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:27.116486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb276c8-4fbb-47c3-99a4-04e422145227 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:27.127317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:27.134222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47db6a-b9b9-4eb2-857f-cf94196244b6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:27.156107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7643b5-630d-49d7-8d28-d514d53d6698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:27.156837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7643b5-630d-49d7-8d28-d514d53d6698 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:27.157181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7643b5-630d-49d7-8d28-d514d53d6698 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:27.157556 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7643b5-630d-49d7-8d28-d514d53d6698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:27.168097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7643b5-630d-49d7-8d28-d514d53d6698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:27.169700 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7643b5-630d-49d7-8d28-d514d53d6698 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:27.171602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:27.174297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626be880-b49b-4046-a859-54da5e47c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:27.175363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be880-b49b-4046-a859-54da5e47c778 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:27.175844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be880-b49b-4046-a859-54da5e47c778 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:27.176317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626be880-b49b-4046-a859-54da5e47c778 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:27.177469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626be880-b49b-4046-a859-54da5e47c778 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:27.177827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626be880-b49b-4046-a859-54da5e47c778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:27.179426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626be880-b49b-4046-a859-54da5e47c778 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-de7643b5-630d-49d7-8d28-d514d53d6698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:27.187812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:27.188933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:27.190106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:27.190106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:27.190106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:27.192402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 343/675] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:27.193591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626be880-b49b-4046-a859-54da5e47c778 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:27.193739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 333/676] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:27.194637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7643b5-630d-49d7-8d28-d514d53d6698 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:27.312658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7643b5-630d-49d7-8d28-d514d53d6698 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:27.316217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626be880-b49b-4046-a859-54da5e47c778 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:27.351633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496f0e0-0ceb-4d0d-9ad2-646eee39ead8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:27.352266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496f0e0-0ceb-4d0d-9ad2-646eee39ead8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:27.352548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496f0e0-0ceb-4d0d-9ad2-646eee39ead8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:27.352889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496f0e0-0ceb-4d0d-9ad2-646eee39ead8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:27.356928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdfe24-41fe-4284-91a3-73c2f0c8eed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:27.358087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdfe24-41fe-4284-91a3-73c2f0c8eed5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:27.358360 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdfe24-41fe-4284-91a3-73c2f0c8eed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:27.358629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdfe24-41fe-4284-91a3-73c2f0c8eed5 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:27.359321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdfe24-41fe-4284-91a3-73c2f0c8eed5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:27.359624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdfe24-41fe-4284-91a3-73c2f0c8eed5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:27.361059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdfe24-41fe-4284-91a3-73c2f0c8eed5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:27.365145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496f0e0-0ceb-4d0d-9ad2-646eee39ead8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a496f0e0-0ceb-4d0d-9ad2-646eee39ead8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:27.370729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:27.371721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 334/677] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:27.372589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496f0e0-0ceb-4d0d-9ad2-646eee39ead8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:27.384874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 344/678] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:27.385693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdfe24-41fe-4284-91a3-73c2f0c8eed5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:27.416817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:27.444564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:27.502004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496f0e0-0ceb-4d0d-9ad2-646eee39ead8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:27.520227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdfe24-41fe-4284-91a3-73c2f0c8eed5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:27.548253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879287a1-0e53-4f15-9cc5-5866bfcb321f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:27.549052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879287a1-0e53-4f15-9cc5-5866bfcb321f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:27.549332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879287a1-0e53-4f15-9cc5-5866bfcb321f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:27.549740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879287a1-0e53-4f15-9cc5-5866bfcb321f None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:27.550442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879287a1-0e53-4f15-9cc5-5866bfcb321f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:27.550746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879287a1-0e53-4f15-9cc5-5866bfcb321f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:27.552183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879287a1-0e53-4f15-9cc5-5866bfcb321f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:27.564190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2269f0-009e-4fb3-9c39-9156e9e36031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:27.565672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2269f0-009e-4fb3-9c39-9156e9e36031 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:27.566121 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2269f0-009e-4fb3-9c39-9156e9e36031 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:27.566535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2269f0-009e-4fb3-9c39-9156e9e36031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:27.566904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 335/679] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:27.567779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879287a1-0e53-4f15-9cc5-5866bfcb321f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:27.583827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2269f0-009e-4fb3-9c39-9156e9e36031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:27.585323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2269f0-009e-4fb3-9c39-9156e9e36031 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bc2269f0-009e-4fb3-9c39-9156e9e36031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:27.602476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:27.603689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:27.605093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:27.605093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:27.605093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:27.609013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 345/680] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:27.609953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2269f0-009e-4fb3-9c39-9156e9e36031 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:27.692138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879287a1-0e53-4f15-9cc5-5866bfcb321f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:27.702645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:27.709149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:27.731604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e2717-ddf4-4af6-bc83-714b4dcc885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:27.732390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e2717-ddf4-4af6-bc83-714b4dcc885a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:27.732768 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e2717-ddf4-4af6-bc83-714b4dcc885a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:27.733150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e2717-ddf4-4af6-bc83-714b4dcc885a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:27.733971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e2717-ddf4-4af6-bc83-714b4dcc885a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:27.734375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e2717-ddf4-4af6-bc83-714b4dcc885a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:27.735654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2269f0-009e-4fb3-9c39-9156e9e36031 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:27.735945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e2717-ddf4-4af6-bc83-714b4dcc885a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:27.751380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 336/681] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:27.751380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e2717-ddf4-4af6-bc83-714b4dcc885a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:27.778438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbb5c2-6d27-477a-b224-565e5b6c2411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:27.779586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbb5c2-6d27-477a-b224-565e5b6c2411 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:27.779884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbb5c2-6d27-477a-b224-565e5b6c2411 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:27.780154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbb5c2-6d27-477a-b224-565e5b6c2411 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:27.781061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbb5c2-6d27-477a-b224-565e5b6c2411 None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=b957b367ddf2483f84056ccec57fc515) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:27.781368 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbb5c2-6d27-477a-b224-565e5b6c2411 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:27.782822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bbb5c2-6d27-477a-b224-565e5b6c2411 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:27.789143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 346/682] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:27 2022] GET /identity/v3/role_assignments?user.id=b957b367ddf2483f84056ccec57fc515&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:27.789971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbb5c2-6d27-477a-b224-565e5b6c2411 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:27.882986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e2717-ddf4-4af6-bc83-714b4dcc885a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:27.923737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419f02e-09e4-42a1-9f7b-31e1744a71cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:27.924383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419f02e-09e4-42a1-9f7b-31e1744a71cf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:27.924778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419f02e-09e4-42a1-9f7b-31e1744a71cf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:27.924928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419f02e-09e4-42a1-9f7b-31e1744a71cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:27.932801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419f02e-09e4-42a1-9f7b-31e1744a71cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:27.934458 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419f02e-09e4-42a1-9f7b-31e1744a71cf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0419f02e-09e4-42a1-9f7b-31e1744a71cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:27.948996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:27.950066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:27.952671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:27.952671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:27.952671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:27.952671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:27.952671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:27.952671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:27.952671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:27.952671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:27.952671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 337/683] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:27.952671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419f02e-09e4-42a1-9f7b-31e1744a71cf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:27.955441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbb5c2-6d27-477a-b224-565e5b6c2411 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:27.991948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:27.999204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d93f25-2790-4b81-8050-08930b047a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:28.000188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d93f25-2790-4b81-8050-08930b047a71 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.000747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d93f25-2790-4b81-8050-08930b047a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.001021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d93f25-2790-4b81-8050-08930b047a71 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.001682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d93f25-2790-4b81-8050-08930b047a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:28.002040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d93f25-2790-4b81-8050-08930b047a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:28.003549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d93f25-2790-4b81-8050-08930b047a71 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:28.005372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:28.020232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 347/684] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:28.021303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d93f25-2790-4b81-8050-08930b047a71 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:28.096920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419f02e-09e4-42a1-9f7b-31e1744a71cf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:28.147875 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e996d5-ea91-4a0d-9ffc-1f49cdcc2d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:28.149008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e996d5-ea91-4a0d-9ffc-1f49cdcc2d39 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.149493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e996d5-ea91-4a0d-9ffc-1f49cdcc2d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.149998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e996d5-ea91-4a0d-9ffc-1f49cdcc2d39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.150205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d93f25-2790-4b81-8050-08930b047a71 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:28.165359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e996d5-ea91-4a0d-9ffc-1f49cdcc2d39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:28.168009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e996d5-ea91-4a0d-9ffc-1f49cdcc2d39 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:28.187265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1857e-cdca-4b2e-a6c6-1cb5f0881576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:28.187882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1857e-cdca-4b2e-a6c6-1cb5f0881576 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.188162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1857e-cdca-4b2e-a6c6-1cb5f0881576 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.188488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1857e-cdca-4b2e-a6c6-1cb5f0881576 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.189195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1857e-cdca-4b2e-a6c6-1cb5f0881576 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:28.189501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1857e-cdca-4b2e-a6c6-1cb5f0881576 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c9e996d5-ea91-4a0d-9ffc-1f49cdcc2d39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:28.190937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:28.191868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:28.192752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:28.192752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:28.192752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:28.192927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df1857e-cdca-4b2e-a6c6-1cb5f0881576 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:28.200249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 338/685] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:28.200249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e996d5-ea91-4a0d-9ffc-1f49cdcc2d39 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:28.210821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 348/686] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:28.211691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1857e-cdca-4b2e-a6c6-1cb5f0881576 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:28.282335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:28.320450 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:28.334999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e996d5-ea91-4a0d-9ffc-1f49cdcc2d39 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:28.360189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1857e-cdca-4b2e-a6c6-1cb5f0881576 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:28.375608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abe495-fdf1-4d8c-8417-80d87716554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:28.376599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abe495-fdf1-4d8c-8417-80d87716554b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.377105 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abe495-fdf1-4d8c-8417-80d87716554b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.377604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4abe495-fdf1-4d8c-8417-80d87716554b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.385831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4abe495-fdf1-4d8c-8417-80d87716554b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:28.387381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4abe495-fdf1-4d8c-8417-80d87716554b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:28.397134 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542589b-c58f-464b-b346-b101b3f52d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:28.397746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542589b-c58f-464b-b346-b101b3f52d92 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.398052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542589b-c58f-464b-b346-b101b3f52d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.398324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542589b-c58f-464b-b346-b101b3f52d92 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.398993 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542589b-c58f-464b-b346-b101b3f52d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:28.399296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542589b-c58f-464b-b346-b101b3f52d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d4abe495-fdf1-4d8c-8417-80d87716554b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:28.399821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:28.400711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:28.401633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:28.401633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:28.401633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:28.401633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:28.401633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:28.401633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:28.401633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:28.401633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:28.401633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:28.401633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:28.402749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542589b-c58f-464b-b346-b101b3f52d92 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:28.402820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 339/687] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:28.402820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abe495-fdf1-4d8c-8417-80d87716554b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:28.416959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 349/688] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:28.416959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d566d5-4abe-4cdb-a334-e6ecd5dbfeff None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.416959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d566d5-4abe-4cdb-a334-e6ecd5dbfeff None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.416959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d566d5-4abe-4cdb-a334-e6ecd5dbfeff None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.421920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 350/689] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:28.425797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d566d5-4abe-4cdb-a334-e6ecd5dbfeff None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:28.518363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4abe495-fdf1-4d8c-8417-80d87716554b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:28.540038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d566d5-4abe-4cdb-a334-e6ecd5dbfeff None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:28.553630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653c3ef-ba7d-452d-b8ce-a9d04a39eb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:28.554271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653c3ef-ba7d-452d-b8ce-a9d04a39eb09 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.554553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653c3ef-ba7d-452d-b8ce-a9d04a39eb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.554829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653c3ef-ba7d-452d-b8ce-a9d04a39eb09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.555555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653c3ef-ba7d-452d-b8ce-a9d04a39eb09 None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=b37a4275e7c6441aa9e8bb0298f56589) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:28.555868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653c3ef-ba7d-452d-b8ce-a9d04a39eb09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:28.557171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:28.557292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653c3ef-ba7d-452d-b8ce-a9d04a39eb09 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:28.573803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 340/690] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:28 2022] GET /identity/v3/role_assignments?user.id=b37a4275e7c6441aa9e8bb0298f56589&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:28.573803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653c3ef-ba7d-452d-b8ce-a9d04a39eb09 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:28.584666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b1d1d-4dd1-4d1f-9817-a44e2e7c06a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:28.585286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b1d1d-4dd1-4d1f-9817-a44e2e7c06a8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.585561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b1d1d-4dd1-4d1f-9817-a44e2e7c06a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.586632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b1d1d-4dd1-4d1f-9817-a44e2e7c06a8 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.587414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b1d1d-4dd1-4d1f-9817-a44e2e7c06a8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:28.587721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b1d1d-4dd1-4d1f-9817-a44e2e7c06a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:28.589808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b1d1d-4dd1-4d1f-9817-a44e2e7c06a8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:28.595426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:28.619391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 351/691] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Aug 19 20:55:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:28.621108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15524d6d-befe-4381-801d-a9932b4d779b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.621393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15524d6d-befe-4381-801d-a9932b4d779b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.621665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15524d6d-befe-4381-801d-a9932b4d779b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.648250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-15524d6d-befe-4381-801d-a9932b4d779b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:28.649522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-15524d6d-befe-4381-801d-a9932b4d779b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:28.724237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653c3ef-ba7d-452d-b8ce-a9d04a39eb09 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:28.733847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 352/692] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:28.734793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15524d6d-befe-4381-801d-a9932b4d779b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:28.768502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b7024-6a86-403a-8d65-b14834ac95d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:28.769368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b7024-6a86-403a-8d65-b14834ac95d3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.769918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b7024-6a86-403a-8d65-b14834ac95d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.770304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b7024-6a86-403a-8d65-b14834ac95d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.782892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b7024-6a86-403a-8d65-b14834ac95d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:28.784353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b7024-6a86-403a-8d65-b14834ac95d3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ce3b7024-6a86-403a-8d65-b14834ac95d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:28.800806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:28.802081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:28.803157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:28.803157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:28.803157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:28.803157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:28.803157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:28.803157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:28.803157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:28.803157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:28.803157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:28.803157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:28.803731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 341/693] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Aug 19 20:55:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:28.809938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b7024-6a86-403a-8d65-b14834ac95d3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:28.876174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15524d6d-befe-4381-801d-a9932b4d779b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:28.890163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:28.900921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:28.921382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae764422-ddf6-4e78-8381-c77d492a4d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:28.922308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae764422-ddf6-4e78-8381-c77d492a4d20 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.922629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae764422-ddf6-4e78-8381-c77d492a4d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.923003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae764422-ddf6-4e78-8381-c77d492a4d20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.946365 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b7024-6a86-403a-8d65-b14834ac95d3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:28.991546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ae3d6-91b4-45da-acff-b7f3c52ebd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:28.992780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ae3d6-91b4-45da-acff-b7f3c52ebd06 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:28.993214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ae3d6-91b4-45da-acff-b7f3c52ebd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:28.994185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1ae3d6-91b4-45da-acff-b7f3c52ebd06 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:28.995233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ae3d6-91b4-45da-acff-b7f3c52ebd06 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:28.995697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ae3d6-91b4-45da-acff-b7f3c52ebd06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:28.998613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1ae3d6-91b4-45da-acff-b7f3c52ebd06 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:29.005010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae764422-ddf6-4e78-8381-c77d492a4d20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:29.006633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae764422-ddf6-4e78-8381-c77d492a4d20 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:29.022558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 342/694] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Aug 19 20:55:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:29.028672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ae3d6-91b4-45da-acff-b7f3c52ebd06 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:29.162845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 353/695] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:29.163992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae764422-ddf6-4e78-8381-c77d492a4d20 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:29.175578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1ae3d6-91b4-45da-acff-b7f3c52ebd06 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:29.190200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:29.216340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:29.223782 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0b12-acaa-4c20-82fa-5a12962110f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:29.224624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0b12-acaa-4c20-82fa-5a12962110f8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:29.225051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0b12-acaa-4c20-82fa-5a12962110f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:29.225883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961f0b12-acaa-4c20-82fa-5a12962110f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:29.288196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae764422-ddf6-4e78-8381-c77d492a4d20 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:29.302520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f0b12-acaa-4c20-82fa-5a12962110f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:29.304257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961f0b12-acaa-4c20-82fa-5a12962110f8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:29.330005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d792c38-3d6d-4abe-bf3c-5f18e68aa055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:29.330779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d792c38-3d6d-4abe-bf3c-5f18e68aa055 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:29.331212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d792c38-3d6d-4abe-bf3c-5f18e68aa055 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:29.331593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d792c38-3d6d-4abe-bf3c-5f18e68aa055 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:29.332281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d792c38-3d6d-4abe-bf3c-5f18e68aa055 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:29.333931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d792c38-3d6d-4abe-bf3c-5f18e68aa055 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6d792c38-3d6d-4abe-bf3c-5f18e68aa055 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:29.349126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:29.350610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 20:55:29.351620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:29.358658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 354/696] 10.222.0.16 () {56 vars in 1129 bytes} [Fri Aug 19 20:55:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:29.358658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d792c38-3d6d-4abe-bf3c-5f18e68aa055 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:29.431671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 343/697] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 19 20:55:29.432933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0b12-acaa-4c20-82fa-5a12962110f8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:29.458294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:29.469170 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:29.485344 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d792c38-3d6d-4abe-bf3c-5f18e68aa055 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:29.533464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8026bdc-b1a0-4197-8665-18ee0cc66135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:29.534188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8026bdc-b1a0-4197-8665-18ee0cc66135 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:29.534464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8026bdc-b1a0-4197-8665-18ee0cc66135 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:29.534733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8026bdc-b1a0-4197-8665-18ee0cc66135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:29.548159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8026bdc-b1a0-4197-8665-18ee0cc66135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:29.549627 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8026bdc-b1a0-4197-8665-18ee0cc66135 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:29.554703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961f0b12-acaa-4c20-82fa-5a12962110f8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b8026bdc-b1a0-4197-8665-18ee0cc66135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:29.566749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:29.567741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:29.568708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:29.568708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:29.568708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:29.568708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 355/698] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:29.569664 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8026bdc-b1a0-4197-8665-18ee0cc66135 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:29.597093 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678aba6-9220-49c9-971a-a91793e6b886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:29.597990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678aba6-9220-49c9-971a-a91793e6b886 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:29.598416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678aba6-9220-49c9-971a-a91793e6b886 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:29.598838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678aba6-9220-49c9-971a-a91793e6b886 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:29.599619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678aba6-9220-49c9-971a-a91793e6b886 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:29.600096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678aba6-9220-49c9-971a-a91793e6b886 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:29.601625 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678aba6-9220-49c9-971a-a91793e6b886 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:29.612282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 344/699] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Aug 19 20:55:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:29.613719 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678aba6-9220-49c9-971a-a91793e6b886 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:29.684580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8026bdc-b1a0-4197-8665-18ee0cc66135 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:29.704829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:29.718362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:29.726051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678aba6-9220-49c9-971a-a91793e6b886 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:29.727391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217df43-80f4-4721-9ef8-3e3487e08ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:29.728971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217df43-80f4-4721-9ef8-3e3487e08ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:29.729484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217df43-80f4-4721-9ef8-3e3487e08ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:29.730018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217df43-80f4-4721-9ef8-3e3487e08ab5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:29.743007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217df43-80f4-4721-9ef8-3e3487e08ab5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:29.745623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217df43-80f4-4721-9ef8-3e3487e08ab5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a217df43-80f4-4721-9ef8-3e3487e08ab5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:29.762325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:29.764024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:29.765569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:29.765569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:29.765569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:29.765740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64458a4-c011-4849-911c-b4a37abb49ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:29.765740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64458a4-c011-4849-911c-b4a37abb49ca None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:29.765740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64458a4-c011-4849-911c-b4a37abb49ca None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:29.765740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64458a4-c011-4849-911c-b4a37abb49ca None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:29.767352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64458a4-c011-4849-911c-b4a37abb49ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:29.767352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64458a4-c011-4849-911c-b4a37abb49ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:29.767352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64458a4-c011-4849-911c-b4a37abb49ca None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:29.769537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 356/700] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:29.770664 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217df43-80f4-4721-9ef8-3e3487e08ab5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:29.786421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 345/701] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:29.787264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64458a4-c011-4849-911c-b4a37abb49ca None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:29.901182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217df43-80f4-4721-9ef8-3e3487e08ab5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:29.906902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64458a4-c011-4849-911c-b4a37abb49ca None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:29.953346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee4f7a7-72b6-419a-94e0-957979bafe46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:29.954548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4f7a7-72b6-419a-94e0-957979bafe46 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:29.954831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239793e-1417-4e33-a788-85acf48e1e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:29.955427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4f7a7-72b6-419a-94e0-957979bafe46 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:29.955576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239793e-1417-4e33-a788-85acf48e1e74 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:29.955792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239793e-1417-4e33-a788-85acf48e1e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:29.955988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239793e-1417-4e33-a788-85acf48e1e74 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:29.956445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee4f7a7-72b6-419a-94e0-957979bafe46 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:29.956628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239793e-1417-4e33-a788-85acf48e1e74 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:29.956934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239793e-1417-4e33-a788-85acf48e1e74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:29.958095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4f7a7-72b6-419a-94e0-957979bafe46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:29.958416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239793e-1417-4e33-a788-85acf48e1e74 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:29.958815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4f7a7-72b6-419a-94e0-957979bafe46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:29.960522 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee4f7a7-72b6-419a-94e0-957979bafe46 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:29.965269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 357/702] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Aug 19 20:55:29 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:29.966165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239793e-1417-4e33-a788-85acf48e1e74 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:29.977078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 346/703] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:29.979341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da12887-fbc7-4508-a0a0-2bde3fafa4e1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:29.980069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da12887-fbc7-4508-a0a0-2bde3fafa4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:29.980573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da12887-fbc7-4508-a0a0-2bde3fafa4e1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:29.984013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 347/704] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:29.985376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da12887-fbc7-4508-a0a0-2bde3fafa4e1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:30.006194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:30.016932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:30.108651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239793e-1417-4e33-a788-85acf48e1e74 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:30.137211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da12887-fbc7-4508-a0a0-2bde3fafa4e1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:30.149300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dafed6-0876-416f-be37-4f06d4c0b5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:30.149979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dafed6-0876-416f-be37-4f06d4c0b5fa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:30.150257 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dafed6-0876-416f-be37-4f06d4c0b5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:30.150527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dafed6-0876-416f-be37-4f06d4c0b5fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:30.161088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dafed6-0876-416f-be37-4f06d4c0b5fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:30.162526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dafed6-0876-416f-be37-4f06d4c0b5fa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c8dafed6-0876-416f-be37-4f06d4c0b5fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:30.176707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:30.177644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:30.178628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:30.178628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:30.178628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:30.178628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:30.178628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:30.178628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:30.178628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:30.178628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:30.179061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 358/705] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:30.180612 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc4153-f1b5-43a0-93da-281f8ac6d8d6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:30.180723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc4153-f1b5-43a0-93da-281f8ac6d8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:30.180958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc4153-f1b5-43a0-93da-281f8ac6d8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:30.191124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175435e-66ae-4c73-9116-3fe7162412d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:30.192251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175435e-66ae-4c73-9116-3fe7162412d7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:30.192842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175435e-66ae-4c73-9116-3fe7162412d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:30.193335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175435e-66ae-4c73-9116-3fe7162412d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:30.202683 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175435e-66ae-4c73-9116-3fe7162412d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:30.204551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175435e-66ae-4c73-9116-3fe7162412d7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:30.212964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-25bc4153-f1b5-43a0-93da-281f8ac6d8d6 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:30.214226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-25bc4153-f1b5-43a0-93da-281f8ac6d8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2175435e-66ae-4c73-9116-3fe7162412d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:30.218438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:30.219394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:30.220261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:30.220261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:30.220261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:30.220261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:30.220261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:30.220261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:30.220261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:30.220261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:30.223925 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 348/706] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:30.225149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175435e-66ae-4c73-9116-3fe7162412d7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:30.297965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 359/707] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:30.298964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc4153-f1b5-43a0-93da-281f8ac6d8d6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:30.315857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:30.353268 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:30.357890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175435e-66ae-4c73-9116-3fe7162412d7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:30.429208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4de10-bf39-41e3-8855-f7336aa8134c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:30.429746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4de10-bf39-41e3-8855-f7336aa8134c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:30.430273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4de10-bf39-41e3-8855-f7336aa8134c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:30.430558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4de10-bf39-41e3-8855-f7336aa8134c None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:30.432563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4de10-bf39-41e3-8855-f7336aa8134c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:30.432883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4de10-bf39-41e3-8855-f7336aa8134c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:30.435288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4de10-bf39-41e3-8855-f7336aa8134c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:30.466083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 349/708] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:30.475359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4de10-bf39-41e3-8855-f7336aa8134c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:30.494452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc4153-f1b5-43a0-93da-281f8ac6d8d6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:30.536339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c8996-14b0-4d6d-811f-061b28b9f2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:30.537544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c8996-14b0-4d6d-811f-061b28b9f2e7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:30.538125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c8996-14b0-4d6d-811f-061b28b9f2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:30.538664 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c8996-14b0-4d6d-811f-061b28b9f2e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:30.539578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c8996-14b0-4d6d-811f-061b28b9f2e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:30.540094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c8996-14b0-4d6d-811f-061b28b9f2e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:30.541914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c8996-14b0-4d6d-811f-061b28b9f2e7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:30.555534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 360/709] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:30.562098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c8996-14b0-4d6d-811f-061b28b9f2e7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:30.597445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4de10-bf39-41e3-8855-f7336aa8134c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:30.639727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc343ed-b042-4db1-b1c0-dea065dbe51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:30.639727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc343ed-b042-4db1-b1c0-dea065dbe51d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:30.639727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc343ed-b042-4db1-b1c0-dea065dbe51d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:30.640102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc343ed-b042-4db1-b1c0-dea065dbe51d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:30.641857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc343ed-b042-4db1-b1c0-dea065dbe51d None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=61a0906b8b134c178b4dece2891f1596) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:30.641970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc343ed-b042-4db1-b1c0-dea065dbe51d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:30.643431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc343ed-b042-4db1-b1c0-dea065dbe51d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:30.651751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:30.657504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 350/710] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:30 2022] GET /identity/v3/role_assignments?user.id=61a0906b8b134c178b4dece2891f1596&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:30.702717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc343ed-b042-4db1-b1c0-dea065dbe51d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:30.729470 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:30.753820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c8996-14b0-4d6d-811f-061b28b9f2e7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:30.848152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc343ed-b042-4db1-b1c0-dea065dbe51d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:30.851576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfb6f6-27d9-4596-8edb-25cf1c759148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:30.852213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfb6f6-27d9-4596-8edb-25cf1c759148 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:30.852553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfb6f6-27d9-4596-8edb-25cf1c759148 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:30.852827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfb6f6-27d9-4596-8edb-25cf1c759148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:30.853502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfb6f6-27d9-4596-8edb-25cf1c759148 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=43a9ad5341b54c40a5427045660363c3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:30.853840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfb6f6-27d9-4596-8edb-25cf1c759148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:30.855394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfb6f6-27d9-4596-8edb-25cf1c759148 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:30.870031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 361/711] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:30 2022] GET /identity/v3/role_assignments?user.id=43a9ad5341b54c40a5427045660363c3&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=8b350845f331423cb0494e3449b58310 => generated 259 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:30.889665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845366a0-7566-4fe3-8846-de6cf3bbf2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:30.890427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845366a0-7566-4fe3-8846-de6cf3bbf2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:30.890752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845366a0-7566-4fe3-8846-de6cf3bbf2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:30.891029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845366a0-7566-4fe3-8846-de6cf3bbf2c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:30.903437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845366a0-7566-4fe3-8846-de6cf3bbf2c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:30.904968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845366a0-7566-4fe3-8846-de6cf3bbf2c5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-845366a0-7566-4fe3-8846-de6cf3bbf2c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:30.921140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:30.923745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:30.925675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:30.925675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:30.925675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:30.925675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 351/712] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:30.931495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845366a0-7566-4fe3-8846-de6cf3bbf2c5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:30.933151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dd387-b666-4a9f-ab44-ec26d02a4ec8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:30.933964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dd387-b666-4a9f-ab44-ec26d02a4ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:30.934834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98dd387-b666-4a9f-ab44-ec26d02a4ec8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:30.947241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 362/713] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:30 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:30.957663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2b0da-8541-4f36-9959-87169fae6b95 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:30.958238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2b0da-8541-4f36-9959-87169fae6b95 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:30.958817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2b0da-8541-4f36-9959-87169fae6b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:30.996753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-22e2b0da-8541-4f36-9959-87169fae6b95 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:30.999677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-22e2b0da-8541-4f36-9959-87169fae6b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:31.003258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:31.085234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845366a0-7566-4fe3-8846-de6cf3bbf2c5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:31.137302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f24437-8527-46d1-8a0e-ba44ec4c6dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:31.137655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f24437-8527-46d1-8a0e-ba44ec4c6dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:31.138003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f24437-8527-46d1-8a0e-ba44ec4c6dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:31.138818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f24437-8527-46d1-8a0e-ba44ec4c6dc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:31.140405 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f24437-8527-46d1-8a0e-ba44ec4c6dc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:31.140405 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f24437-8527-46d1-8a0e-ba44ec4c6dc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:31.141491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f24437-8527-46d1-8a0e-ba44ec4c6dc1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:31.142983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 363/714] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:31.157722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 352/715] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:31.163279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2b0da-8541-4f36-9959-87169fae6b95 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:31.272059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:31.298961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2b0da-8541-4f36-9959-87169fae6b95 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:31.349576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f2370-7c2f-4ac3-bf44-cecab2e14e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:31.350272 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f2370-7c2f-4ac3-bf44-cecab2e14e6b None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:31.350558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f2370-7c2f-4ac3-bf44-cecab2e14e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:31.351024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f2370-7c2f-4ac3-bf44-cecab2e14e6b None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:31.351744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f2370-7c2f-4ac3-bf44-cecab2e14e6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:31.353467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f2370-7c2f-4ac3-bf44-cecab2e14e6b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:31.394649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 364/716] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:55:31 2022] POST /identity/v3/users => generated 254 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:31.462754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f24437-8527-46d1-8a0e-ba44ec4c6dc1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:31.582710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f24437-8527-46d1-8a0e-ba44ec4c6dc1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:31.598292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:31.627934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060abd-8e9e-490d-a056-6fa1055aca3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:31.628587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060abd-8e9e-490d-a056-6fa1055aca3f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:31.628865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060abd-8e9e-490d-a056-6fa1055aca3f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:31.629134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa060abd-8e9e-490d-a056-6fa1055aca3f None admin] PATH_INFO: `/v3/services/network` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:31.629733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa060abd-8e9e-490d-a056-6fa1055aca3f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:31.631149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa060abd-8e9e-490d-a056-6fa1055aca3f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-aa060abd-8e9e-490d-a056-6fa1055aca3f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:31.644149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:31.645184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:31.646028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 353/717] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:31 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:31.655492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f2370-7c2f-4ac3-bf44-cecab2e14e6b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:31.783253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f2370-7c2f-4ac3-bf44-cecab2e14e6b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:31.821176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3099fda-cc8d-4817-b3f7-ce7913974821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:31.821799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3099fda-cc8d-4817-b3f7-ce7913974821 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:31.822080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3099fda-cc8d-4817-b3f7-ce7913974821 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:31.822352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3099fda-cc8d-4817-b3f7-ce7913974821 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:31.823048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3099fda-cc8d-4817-b3f7-ce7913974821 None admin] RBAC: Adding query filter params (name=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:31.823352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3099fda-cc8d-4817-b3f7-ce7913974821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:31.824719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3099fda-cc8d-4817-b3f7-ce7913974821 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:31.827014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:31.832271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 365/718] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:55:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:31.837722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060abd-8e9e-490d-a056-6fa1055aca3f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:31.947750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa060abd-8e9e-490d-a056-6fa1055aca3f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:31.985818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684bdc4-061a-4b4c-8977-86b5375c2e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:31.986441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684bdc4-061a-4b4c-8977-86b5375c2e8a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:31.986716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684bdc4-061a-4b4c-8977-86b5375c2e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:31.986990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684bdc4-061a-4b4c-8977-86b5375c2e8a None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:31.987625 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684bdc4-061a-4b4c-8977-86b5375c2e8a None admin] RBAC: Adding query filter params (type=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:31.987928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684bdc4-061a-4b4c-8977-86b5375c2e8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:31.989307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684bdc4-061a-4b4c-8977-86b5375c2e8a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:31.995884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 354/719] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:55:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:32.402141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d168a2-e706-45da-a32f-a444f96e5669 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:32.402553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d168a2-e706-45da-a32f-a444f96e5669 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:32.402755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d168a2-e706-45da-a32f-a444f96e5669 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:32.404568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 366/720] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:32.410616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8cc8b-2213-4318-959d-80db4c5a4976 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:32.410891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8cc8b-2213-4318-959d-80db4c5a4976 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:32.411172 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8cc8b-2213-4318-959d-80db4c5a4976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:32.429977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c8c8cc8b-2213-4318-959d-80db4c5a4976 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:32.431187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c8c8cc8b-2213-4318-959d-80db4c5a4976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:32.464052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:32.521310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 355/721] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:32.909091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d168a2-e706-45da-a32f-a444f96e5669 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:33.059135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d168a2-e706-45da-a32f-a444f96e5669 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:33.109981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c6247-2ef3-4656-af68-0ad8d31d993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:33.110636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c6247-2ef3-4656-af68-0ad8d31d993d None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:33.110913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c6247-2ef3-4656-af68-0ad8d31d993d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:33.111175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c6247-2ef3-4656-af68-0ad8d31d993d None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:33.111824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c6247-2ef3-4656-af68-0ad8d31d993d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:33.113252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c6247-2ef3-4656-af68-0ad8d31d993d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:33.144298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 367/722] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:55:32 2022] POST /identity/v3/services => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:33.207543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299cfa0-d63d-4ec7-a191-4c8801a5a78a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:33.208066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299cfa0-d63d-4ec7-a191-4c8801a5a78a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:33.208066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299cfa0-d63d-4ec7-a191-4c8801a5a78a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:33.209716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 356/723] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:33.215565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd6935-4f48-4fa0-9dc4-67bd7569ec84 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:33.215849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd6935-4f48-4fa0-9dc4-67bd7569ec84 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:33.216140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd6935-4f48-4fa0-9dc4-67bd7569ec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:33.247069 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e8bd6935-4f48-4fa0-9dc4-67bd7569ec84 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:33.249149 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e8bd6935-4f48-4fa0-9dc4-67bd7569ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:33.333287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:33.357542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 368/724] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:33.637944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299cfa0-d63d-4ec7-a191-4c8801a5a78a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:33.697492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84ee4a-5a8e-4fbb-a2f4-a3cac92a52a6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:33.697861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84ee4a-5a8e-4fbb-a2f4-a3cac92a52a6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:33.698058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84ee4a-5a8e-4fbb-a2f4-a3cac92a52a6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:33.699691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 369/725] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:33.705554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec407cb-67bf-4701-ab6e-e4aa83b95eaf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:33.705895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec407cb-67bf-4701-ab6e-e4aa83b95eaf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:33.706179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec407cb-67bf-4701-ab6e-e4aa83b95eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:33.726324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9ec407cb-67bf-4701-ab6e-e4aa83b95eaf None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:33.727606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9ec407cb-67bf-4701-ab6e-e4aa83b95eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:33.778185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299cfa0-d63d-4ec7-a191-4c8801a5a78a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:33.846450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 370/726] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:55:33.861591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282e2d6-754c-4ca0-9be5-986bd854091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:33.862856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282e2d6-754c-4ca0-9be5-986bd854091b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:33.863441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282e2d6-754c-4ca0-9be5-986bd854091b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:33.863927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282e2d6-754c-4ca0-9be5-986bd854091b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:33.888115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e282e2d6-754c-4ca0-9be5-986bd854091b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e282e2d6-754c-4ca0-9be5-986bd854091b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:33.893339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:33.894329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:33.895143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:33.899003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 357/727] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:33.907028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec407cb-67bf-4701-ab6e-e4aa83b95eaf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:34.068775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec407cb-67bf-4701-ab6e-e4aa83b95eaf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:34.099042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:34.129903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860cb0f-c08c-4f5a-a888-5d597b889f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:34.130528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860cb0f-c08c-4f5a-a888-5d597b889f50 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:34.130809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860cb0f-c08c-4f5a-a888-5d597b889f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:34.131086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860cb0f-c08c-4f5a-a888-5d597b889f50 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:34.131767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860cb0f-c08c-4f5a-a888-5d597b889f50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:34.132071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860cb0f-c08c-4f5a-a888-5d597b889f50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:34.133467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3860cb0f-c08c-4f5a-a888-5d597b889f50 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:34.145307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 371/728] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:34.150445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282e2d6-754c-4ca0-9be5-986bd854091b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:34.153492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9753-96ea-48f1-b287-f836ee464970 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:34.153799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9753-96ea-48f1-b287-f836ee464970 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:34.154699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bf9753-96ea-48f1-b287-f836ee464970 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:34.156551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 372/729] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:55:34.162092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fbf9f-bced-4aa8-8a5f-8af24053727a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:34.162092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fbf9f-bced-4aa8-8a5f-8af24053727a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:34.162464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fbf9f-bced-4aa8-8a5f-8af24053727a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:34.191356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-337fbf9f-bced-4aa8-8a5f-8af24053727a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:34.192586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-337fbf9f-bced-4aa8-8a5f-8af24053727a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:34.211441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:34.296522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 373/730] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 19 20:55:34.297999 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fbf9f-bced-4aa8-8a5f-8af24053727a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:34.312046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282e2d6-754c-4ca0-9be5-986bd854091b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:34.361195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc43c67-9049-4b78-be4c-a23753d7cff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:34.362238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc43c67-9049-4b78-be4c-a23753d7cff3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:34.363970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc43c67-9049-4b78-be4c-a23753d7cff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:34.364673 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc43c67-9049-4b78-be4c-a23753d7cff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:34.386675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc43c67-9049-4b78-be4c-a23753d7cff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:34.390029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc43c67-9049-4b78-be4c-a23753d7cff3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5fc43c67-9049-4b78-be4c-a23753d7cff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:34.418221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:34.419188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:34.420277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:34.420277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:34.420277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:34.430063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 358/731] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:34.435792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc43c67-9049-4b78-be4c-a23753d7cff3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:34.453382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fbf9f-bced-4aa8-8a5f-8af24053727a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:34.469442 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:34.531884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c633c-a22a-46e1-babc-7d955d78977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:34.532805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c633c-a22a-46e1-babc-7d955d78977b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:34.533956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c633c-a22a-46e1-babc-7d955d78977b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:34.534389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c633c-a22a-46e1-babc-7d955d78977b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:34.546319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c633c-a22a-46e1-babc-7d955d78977b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b45c633c-a22a-46e1-babc-7d955d78977b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:34.551362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:34.552361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:34.553241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:34.556977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 374/732] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:34.558544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09591ce1-123c-45e8-9927-0a28712b471f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:34.559087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09591ce1-123c-45e8-9927-0a28712b471f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:34.559576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09591ce1-123c-45e8-9927-0a28712b471f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:34.561912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 375/733] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:34.566961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ac8d0-05d4-4f0a-a6e7-d6b44c558d8a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:34.567240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ac8d0-05d4-4f0a-a6e7-d6b44c558d8a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:34.567510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ac8d0-05d4-4f0a-a6e7-d6b44c558d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:34.571987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:34.588660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1f5ac8d0-05d4-4f0a-a6e7-d6b44c558d8a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:34.589931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1f5ac8d0-05d4-4f0a-a6e7-d6b44c558d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:34.601895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc43c67-9049-4b78-be4c-a23753d7cff3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:34.638984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c150950-1552-45b6-a685-0ca13f906d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:34.639501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c150950-1552-45b6-a685-0ca13f906d37 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:34.639782 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c150950-1552-45b6-a685-0ca13f906d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:34.640062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c150950-1552-45b6-a685-0ca13f906d37 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:34.640722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c150950-1552-45b6-a685-0ca13f906d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:34.641107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c150950-1552-45b6-a685-0ca13f906d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:34.642451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c150950-1552-45b6-a685-0ca13f906d37 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:34.655267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 359/734] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:34.656117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c150950-1552-45b6-a685-0ca13f906d37 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:34.675384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 376/735] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 19 20:55:34.676452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ac8d0-05d4-4f0a-a6e7-d6b44c558d8a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:34.782110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c150950-1552-45b6-a685-0ca13f906d37 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:34.805175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:34.811899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ac8d0-05d4-4f0a-a6e7-d6b44c558d8a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:34.821525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937d696-4b69-4e9d-9587-578d54d525e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:34.822589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d696-4b69-4e9d-9587-578d54d525e9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:34.823112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d696-4b69-4e9d-9587-578d54d525e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:34.823609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d696-4b69-4e9d-9587-578d54d525e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:34.824512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937d696-4b69-4e9d-9587-578d54d525e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:34.825040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937d696-4b69-4e9d-9587-578d54d525e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:34.827279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937d696-4b69-4e9d-9587-578d54d525e9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:34.837064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 360/736] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:34.838256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937d696-4b69-4e9d-9587-578d54d525e9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:34.843871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:34.855237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae4497-7755-41da-9e9a-b0d59af31d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:34.855809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae4497-7755-41da-9e9a-b0d59af31d06 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:34.856095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae4497-7755-41da-9e9a-b0d59af31d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:34.856371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae4497-7755-41da-9e9a-b0d59af31d06 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:34.869117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae4497-7755-41da-9e9a-b0d59af31d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f9ae4497-7755-41da-9e9a-b0d59af31d06 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:34.874349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:34.875269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:34.876112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 377/737] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:34.876817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae4497-7755-41da-9e9a-b0d59af31d06 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:34.959304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937d696-4b69-4e9d-9587-578d54d525e9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:34.992458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae4497-7755-41da-9e9a-b0d59af31d06 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:34.999214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b028-98cb-408c-ac2c-6c2c4488bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:34.999801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b028-98cb-408c-ac2c-6c2c4488bc8c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.000078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b028-98cb-408c-ac2c-6c2c4488bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.000350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c608b028-98cb-408c-ac2c-6c2c4488bc8c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.009772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b028-98cb-408c-ac2c-6c2c4488bc8c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:35.011238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c608b028-98cb-408c-ac2c-6c2c4488bc8c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c608b028-98cb-408c-ac2c-6c2c4488bc8c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:35.023991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:35.025053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:35.026040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 361/738] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Aug 19 20:55:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:35.026650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b028-98cb-408c-ac2c-6c2c4488bc8c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:35.038389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79ef5c-2571-442e-93fc-996b36819f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:35.039618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79ef5c-2571-442e-93fc-996b36819f33 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.040067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79ef5c-2571-442e-93fc-996b36819f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.040796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79ef5c-2571-442e-93fc-996b36819f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.055907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79ef5c-2571-442e-93fc-996b36819f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:35.058190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79ef5c-2571-442e-93fc-996b36819f33 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4b79ef5c-2571-442e-93fc-996b36819f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:35.077659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:35.079271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:35.080252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:35.080252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:35.080252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:35.087224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 378/739] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:35.087224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79ef5c-2571-442e-93fc-996b36819f33 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:35.097433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:35.099270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:35.142550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c608b028-98cb-408c-ac2c-6c2c4488bc8c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:35.177659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbe434-1993-4152-8061-47eb9d749df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:35.178612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbe434-1993-4152-8061-47eb9d749df7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.178897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbe434-1993-4152-8061-47eb9d749df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.179174 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbe434-1993-4152-8061-47eb9d749df7 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.179851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbe434-1993-4152-8061-47eb9d749df7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:35.180153 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbe434-1993-4152-8061-47eb9d749df7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:35.181543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fbe434-1993-4152-8061-47eb9d749df7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:35.190870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 362/740] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:35.191732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbe434-1993-4152-8061-47eb9d749df7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:35.211910 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79ef5c-2571-442e-93fc-996b36819f33 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:35.250826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0261cd5-e396-4de3-a469-3ee4a81ceee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:35.252001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0261cd5-e396-4de3-a469-3ee4a81ceee0 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.252606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0261cd5-e396-4de3-a469-3ee4a81ceee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.253055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0261cd5-e396-4de3-a469-3ee4a81ceee0 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.254445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0261cd5-e396-4de3-a469-3ee4a81ceee0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:35.256488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0261cd5-e396-4de3-a469-3ee4a81ceee0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:35.277939 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 379/741] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:55:35 2022] POST /identity/v3/services => generated 240 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:35.279743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0261cd5-e396-4de3-a469-3ee4a81ceee0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:35.314326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbe434-1993-4152-8061-47eb9d749df7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:35.350841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:35.375497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:35.388100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58ba05-d012-4ba1-ae6d-7fec47f59303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:35.390122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58ba05-d012-4ba1-ae6d-7fec47f59303 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.390638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58ba05-d012-4ba1-ae6d-7fec47f59303 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.391071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58ba05-d012-4ba1-ae6d-7fec47f59303 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.392608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58ba05-d012-4ba1-ae6d-7fec47f59303 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:35.393092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58ba05-d012-4ba1-ae6d-7fec47f59303 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:35.395291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58ba05-d012-4ba1-ae6d-7fec47f59303 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:35.421292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0261cd5-e396-4de3-a469-3ee4a81ceee0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:35.423747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 363/742] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Aug 19 20:55:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:35.425798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58ba05-d012-4ba1-ae6d-7fec47f59303 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:35.466924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821457a6-8c5c-43c5-94b1-34276acd7c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:35.468270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821457a6-8c5c-43c5-94b1-34276acd7c7c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.469812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821457a6-8c5c-43c5-94b1-34276acd7c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.470946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821457a6-8c5c-43c5-94b1-34276acd7c7c None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.471853 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821457a6-8c5c-43c5-94b1-34276acd7c7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:35.473000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821457a6-8c5c-43c5-94b1-34276acd7c7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:35.474855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821457a6-8c5c-43c5-94b1-34276acd7c7c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:35.494052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 380/743] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:35.494052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821457a6-8c5c-43c5-94b1-34276acd7c7c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:35.583317 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58ba05-d012-4ba1-ae6d-7fec47f59303 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:35.613682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821457a6-8c5c-43c5-94b1-34276acd7c7c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:35.623431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80845196-d3dc-433f-8a9c-95184bc1ff95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:35.624137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80845196-d3dc-433f-8a9c-95184bc1ff95 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.624448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80845196-d3dc-433f-8a9c-95184bc1ff95 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.624720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80845196-d3dc-433f-8a9c-95184bc1ff95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.637108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80845196-d3dc-433f-8a9c-95184bc1ff95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:35.639108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80845196-d3dc-433f-8a9c-95184bc1ff95 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:35.645370 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-80845196-d3dc-433f-8a9c-95184bc1ff95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:35.655169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:35.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:35.657691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:35.657691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:35.657691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:35.657897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ec492-d080-4d1b-8afd-7aa1217f1c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:35.657897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ec492-d080-4d1b-8afd-7aa1217f1c95 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.658089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 364/744] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:35.658089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80845196-d3dc-433f-8a9c-95184bc1ff95 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:35.661119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ec492-d080-4d1b-8afd-7aa1217f1c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.661448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ec492-d080-4d1b-8afd-7aa1217f1c95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.673153 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ec492-d080-4d1b-8afd-7aa1217f1c95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:35.674647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ec492-d080-4d1b-8afd-7aa1217f1c95 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b65ec492-d080-4d1b-8afd-7aa1217f1c95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:35.690372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:35.691398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:35.692873 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:35.692873 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:35.692873 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:35.692873 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 381/745] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:35.693115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e8877-4494-48fa-8ac3-51067a356073 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.693424 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e8877-4494-48fa-8ac3-51067a356073 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.693531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e8877-4494-48fa-8ac3-51067a356073 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.694966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 382/746] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:35.695505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e8877-4494-48fa-8ac3-51067a356073 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:35.701672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:35.781699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80845196-d3dc-433f-8a9c-95184bc1ff95 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:35.813034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e8877-4494-48fa-8ac3-51067a356073 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:35.825213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58263d7f-306a-43d5-8113-6df3b542a837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:35.826835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58263d7f-306a-43d5-8113-6df3b542a837 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.827122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58263d7f-306a-43d5-8113-6df3b542a837 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.827777 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58263d7f-306a-43d5-8113-6df3b542a837 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.837986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58263d7f-306a-43d5-8113-6df3b542a837 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:35.839962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58263d7f-306a-43d5-8113-6df3b542a837 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:35.853969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a911cc8-33b6-43cf-b9b8-5926326aee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:35.854577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911cc8-33b6-43cf-b9b8-5926326aee3b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.854858 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911cc8-33b6-43cf-b9b8-5926326aee3b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.855138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a911cc8-33b6-43cf-b9b8-5926326aee3b None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-58263d7f-306a-43d5-8113-6df3b542a837 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:35.855636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:35.856573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:35.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:35.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:35.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:35.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:35.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:35.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:35.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:35.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:35.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:35.857533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:35.863811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a911cc8-33b6-43cf-b9b8-5926326aee3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:35.863811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a911cc8-33b6-43cf-b9b8-5926326aee3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:35.863962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 365/747] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:35.863962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58263d7f-306a-43d5-8113-6df3b542a837 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:35.864098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a911cc8-33b6-43cf-b9b8-5926326aee3b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:35.873382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 383/748] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:35.875780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29347-9d4f-4ee0-9838-ef737682d4f6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:35.876296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29347-9d4f-4ee0-9838-ef737682d4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:35.877149 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b29347-9d4f-4ee0-9838-ef737682d4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:35.897367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-45b29347-9d4f-4ee0-9838-ef737682d4f6 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:35.898613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-45b29347-9d4f-4ee0-9838-ef737682d4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:35.938561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:35.964667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:35.976432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58263d7f-306a-43d5-8113-6df3b542a837 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:35.983542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 384/749] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:35.990276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b29347-9d4f-4ee0-9838-ef737682d4f6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:36.027435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10798946-cb98-492d-a3aa-4a1f85fa94fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:36.027435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10798946-cb98-492d-a3aa-4a1f85fa94fb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:36.027435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10798946-cb98-492d-a3aa-4a1f85fa94fb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:36.027435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10798946-cb98-492d-a3aa-4a1f85fa94fb None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:36.028030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10798946-cb98-492d-a3aa-4a1f85fa94fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:36.028406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10798946-cb98-492d-a3aa-4a1f85fa94fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:36.030078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10798946-cb98-492d-a3aa-4a1f85fa94fb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:36.047854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 366/750] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:36.048842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10798946-cb98-492d-a3aa-4a1f85fa94fb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:36.120526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b29347-9d4f-4ee0-9838-ef737682d4f6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:36.157313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e5042-4515-49ce-970a-73e4e71770fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:36.159245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e5042-4515-49ce-970a-73e4e71770fe None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:36.159708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e5042-4515-49ce-970a-73e4e71770fe None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:36.159819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10798946-cb98-492d-a3aa-4a1f85fa94fb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:36.160222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e5042-4515-49ce-970a-73e4e71770fe None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:36.161101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e5042-4515-49ce-970a-73e4e71770fe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:36.161544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e5042-4515-49ce-970a-73e4e71770fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:36.163156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e5042-4515-49ce-970a-73e4e71770fe None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:36.187341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 385/751] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Aug 19 20:55:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:36.188514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e5042-4515-49ce-970a-73e4e71770fe None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:36.198913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff6b92-2804-4e98-ac36-e802b44b39f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:36.199657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff6b92-2804-4e98-ac36-e802b44b39f4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:36.200045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff6b92-2804-4e98-ac36-e802b44b39f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:36.200392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff6b92-2804-4e98-ac36-e802b44b39f4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:36.209203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff6b92-2804-4e98-ac36-e802b44b39f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:36.210845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff6b92-2804-4e98-ac36-e802b44b39f4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:36.223573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:36.319758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e5042-4515-49ce-970a-73e4e71770fe None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d5ff6b92-2804-4e98-ac36-e802b44b39f4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:36.340636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:36.342985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 367/752] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Aug 19 20:55:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:36.343933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff6b92-2804-4e98-ac36-e802b44b39f4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:36.359720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:36.366511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d174322-d51f-438e-bac0-722d73c69461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:36.367151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d174322-d51f-438e-bac0-722d73c69461 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:36.367429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d174322-d51f-438e-bac0-722d73c69461 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:36.367703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d174322-d51f-438e-bac0-722d73c69461 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:36.375916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d174322-d51f-438e-bac0-722d73c69461 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:36.377383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d174322-d51f-438e-bac0-722d73c69461 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0d174322-d51f-438e-bac0-722d73c69461 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:36.389976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:36.391112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:36.392208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:36.392208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:36.392208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:36.392208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:36.392208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:36.392208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:36.392208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:36.392208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:36.395831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 386/753] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:36.396716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d174322-d51f-438e-bac0-722d73c69461 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:36.466392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff6b92-2804-4e98-ac36-e802b44b39f4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:36.498809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:36.505397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c83b8e-a6f4-4920-9b40-dd0b735f7cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:36.506057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c83b8e-a6f4-4920-9b40-dd0b735f7cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:36.506340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c83b8e-a6f4-4920-9b40-dd0b735f7cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:36.506614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c83b8e-a6f4-4920-9b40-dd0b735f7cf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:36.514482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d174322-d51f-438e-bac0-722d73c69461 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:36.558207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadc4ce-a476-4372-9e92-f515e1490903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:36.559231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadc4ce-a476-4372-9e92-f515e1490903 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:36.559714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadc4ce-a476-4372-9e92-f515e1490903 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:36.560791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eadc4ce-a476-4372-9e92-f515e1490903 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:36.561730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eadc4ce-a476-4372-9e92-f515e1490903 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:36.563400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eadc4ce-a476-4372-9e92-f515e1490903 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3eadc4ce-a476-4372-9e92-f515e1490903 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:36.573203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:36.574376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:36.575242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 387/754] 10.222.0.16 () {56 vars in 1129 bytes} [Fri Aug 19 20:55:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:36.576096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadc4ce-a476-4372-9e92-f515e1490903 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:36.598660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c83b8e-a6f4-4920-9b40-dd0b735f7cf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:36.600161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c83b8e-a6f4-4920-9b40-dd0b735f7cf8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:36.613816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:36.697688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eadc4ce-a476-4372-9e92-f515e1490903 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:36.719627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 368/755] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:55:36.720715 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c83b8e-a6f4-4920-9b40-dd0b735f7cf8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:36.744865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46ef4c-71d0-487e-ba96-4280a228cbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:36.745491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46ef4c-71d0-487e-ba96-4280a228cbe5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:36.745802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46ef4c-71d0-487e-ba96-4280a228cbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:36.746101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46ef4c-71d0-487e-ba96-4280a228cbe5 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:36.746819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46ef4c-71d0-487e-ba96-4280a228cbe5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:36.747126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46ef4c-71d0-487e-ba96-4280a228cbe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:36.749583 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46ef4c-71d0-487e-ba96-4280a228cbe5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:36.773393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 388/756] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Aug 19 20:55:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:36.775045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46ef4c-71d0-487e-ba96-4280a228cbe5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:36.788295 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:36.839143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c83b8e-a6f4-4920-9b40-dd0b735f7cf8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:36.852128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:36.879619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be933d48-a8ea-480f-a687-d72eb4576487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:36.880638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be933d48-a8ea-480f-a687-d72eb4576487 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:36.881118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be933d48-a8ea-480f-a687-d72eb4576487 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:36.881563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be933d48-a8ea-480f-a687-d72eb4576487 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:36.882565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be933d48-a8ea-480f-a687-d72eb4576487 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:36.883187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be933d48-a8ea-480f-a687-d72eb4576487 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:36.884632 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be933d48-a8ea-480f-a687-d72eb4576487 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:36.888857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46ef4c-71d0-487e-ba96-4280a228cbe5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:36.902612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 369/757] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:36.902612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be933d48-a8ea-480f-a687-d72eb4576487 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:36.931573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d7a57-df31-493e-bdfc-b2476afc0eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:36.932193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d7a57-df31-493e-bdfc-b2476afc0eea None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:36.932472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d7a57-df31-493e-bdfc-b2476afc0eea None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:36.932755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d7a57-df31-493e-bdfc-b2476afc0eea None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:36.933387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d7a57-df31-493e-bdfc-b2476afc0eea None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:36.933692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d7a57-df31-493e-bdfc-b2476afc0eea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:36.935704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d7a57-df31-493e-bdfc-b2476afc0eea None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:36.940643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 389/758] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Aug 19 20:55:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:36.941465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d7a57-df31-493e-bdfc-b2476afc0eea None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:37.016047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be933d48-a8ea-480f-a687-d72eb4576487 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:37.032597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:37.052388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced600d-fea7-46d4-9913-a5d959fbd009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:37.053020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced600d-fea7-46d4-9913-a5d959fbd009 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.053298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced600d-fea7-46d4-9913-a5d959fbd009 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.053576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced600d-fea7-46d4-9913-a5d959fbd009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.062045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d7a57-df31-493e-bdfc-b2476afc0eea None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:37.066192 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced600d-fea7-46d4-9913-a5d959fbd009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:37.067662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced600d-fea7-46d4-9913-a5d959fbd009 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8ced600d-fea7-46d4-9913-a5d959fbd009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:37.082774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:37.083755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:37.084722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:37.084722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:37.084722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:37.088828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 370/759] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:37.089813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced600d-fea7-46d4-9913-a5d959fbd009 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:37.103195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d935ff-ec3e-415c-8736-4078a69561bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:37.103789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d935ff-ec3e-415c-8736-4078a69561bb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.104056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d935ff-ec3e-415c-8736-4078a69561bb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.104336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d935ff-ec3e-415c-8736-4078a69561bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.117949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:37.176645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d935ff-ec3e-415c-8736-4078a69561bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:37.178129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d935ff-ec3e-415c-8736-4078a69561bb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:37.208451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced600d-fea7-46d4-9913-a5d959fbd009 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:37.245963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bd332-b895-45bd-8b07-a7e66b83e80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:37.246628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bd332-b895-45bd-8b07-a7e66b83e80c None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.246905 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bd332-b895-45bd-8b07-a7e66b83e80c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.247180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bd332-b895-45bd-8b07-a7e66b83e80c None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/43a9ad5341b54c40a5427045660363c3/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.249325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bd332-b895-45bd-8b07-a7e66b83e80c None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=43a9ad5341b54c40a5427045660363c3, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:37.250796 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bd332-b895-45bd-8b07-a7e66b83e80c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:37.261211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:37.266639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 371/760] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:55:37 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/43a9ad5341b54c40a5427045660363c3/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:55:37.267695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bd332-b895-45bd-8b07-a7e66b83e80c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:37.303245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 390/761] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:55:37.304263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d935ff-ec3e-415c-8736-4078a69561bb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:37.369758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:37.391444 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bd332-b895-45bd-8b07-a7e66b83e80c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:37.424850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d935ff-ec3e-415c-8736-4078a69561bb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:37.433686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9755953-1611-43bd-8316-bb4f7b6300b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:37.434330 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9755953-1611-43bd-8316-bb4f7b6300b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.434605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9755953-1611-43bd-8316-bb4f7b6300b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.434875 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9755953-1611-43bd-8316-bb4f7b6300b3 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.436103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9755953-1611-43bd-8316-bb4f7b6300b3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:37.436587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9755953-1611-43bd-8316-bb4f7b6300b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:37.438009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9755953-1611-43bd-8316-bb4f7b6300b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:37.443686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 372/762] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Aug 19 20:55:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:37.445011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbb32-56c9-45c7-b4c3-f964d145c2f8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.445289 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbb32-56c9-45c7-b4c3-f964d145c2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.445560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbb32-56c9-45c7-b4c3-f964d145c2f8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.449943 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 373/763] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:37.451038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdbb32-56c9-45c7-b4c3-f964d145c2f8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:37.467077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33e30f-68df-42b9-9494-3cbbd1748b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:37.467981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33e30f-68df-42b9-9494-3cbbd1748b4b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.468450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33e30f-68df-42b9-9494-3cbbd1748b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.468721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33e30f-68df-42b9-9494-3cbbd1748b4b None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.469364 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33e30f-68df-42b9-9494-3cbbd1748b4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:37.469667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33e30f-68df-42b9-9494-3cbbd1748b4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:37.471065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33e30f-68df-42b9-9494-3cbbd1748b4b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:37.483521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 391/764] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:37.484617 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33e30f-68df-42b9-9494-3cbbd1748b4b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:37.525158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:37.565209 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdbb32-56c9-45c7-b4c3-f964d145c2f8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:37.598871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33e30f-68df-42b9-9494-3cbbd1748b4b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:37.604267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5e0cf9-378c-41da-bd8d-7007776d8ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:37.604903 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e0cf9-378c-41da-bd8d-7007776d8ffb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.605179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e0cf9-378c-41da-bd8d-7007776d8ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.605452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e0cf9-378c-41da-bd8d-7007776d8ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.618266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e0cf9-378c-41da-bd8d-7007776d8ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:37.620321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5e0cf9-378c-41da-bd8d-7007776d8ffb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:37.629290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-cf5e0cf9-378c-41da-bd8d-7007776d8ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:37.637861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:37.639835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:37.640723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:37.640723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:37.640723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:37.640723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 374/765] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:37.640723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef2c9c-5561-40db-8f36-fc870cd82c2a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.640989 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef2c9c-5561-40db-8f36-fc870cd82c2a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.641578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ef2c9c-5561-40db-8f36-fc870cd82c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.645716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fb7c8-00ba-4876-a27a-3652f6097f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:37.646408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fb7c8-00ba-4876-a27a-3652f6097f90 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.646689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fb7c8-00ba-4876-a27a-3652f6097f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.646957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7fb7c8-00ba-4876-a27a-3652f6097f90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.647846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fb7c8-00ba-4876-a27a-3652f6097f90 None admin] RBAC: Adding query filter params (interface=public, service_id=4bd3301dae164dbb8e81c3d1fdba5907, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:37.648161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fb7c8-00ba-4876-a27a-3652f6097f90 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:37.650473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7fb7c8-00ba-4876-a27a-3652f6097f90 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:37.659992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 392/766] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Aug 19 20:55:37 2022] GET /identity/v3/endpoints?service_id=4bd3301dae164dbb8e81c3d1fdba5907&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:37.660838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fb7c8-00ba-4876-a27a-3652f6097f90 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:37.662537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-56ef2c9c-5561-40db-8f36-fc870cd82c2a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:37.663817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-56ef2c9c-5561-40db-8f36-fc870cd82c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:37.740739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 375/767] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:37.741716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef2c9c-5561-40db-8f36-fc870cd82c2a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:37.777733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:37.782878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7fb7c8-00ba-4876-a27a-3652f6097f90 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:37.823137 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb7e97-7cd1-4abf-aefc-1f98b940d735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:37.824074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb7e97-7cd1-4abf-aefc-1f98b940d735 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.824503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb7e97-7cd1-4abf-aefc-1f98b940d735 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.824922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb7e97-7cd1-4abf-aefc-1f98b940d735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.832883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb7e97-7cd1-4abf-aefc-1f98b940d735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:37.834579 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb7e97-7cd1-4abf-aefc-1f98b940d735 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7ceb7e97-7cd1-4abf-aefc-1f98b940d735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:37.846694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:37.847626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:37.848577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:37.848577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:37.848577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:37.848577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:37.848577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:37.848577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:37.848577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:37.848577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:37.850951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 393/768] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:37.852001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb7e97-7cd1-4abf-aefc-1f98b940d735 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:37.887775 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ef2c9c-5561-40db-8f36-fc870cd82c2a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:37.919528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:37.933985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1337d-e7ce-4790-a55d-bcfb8f76252a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:37.934962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1337d-e7ce-4790-a55d-bcfb8f76252a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:37.935579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1337d-e7ce-4790-a55d-bcfb8f76252a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:37.936046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1337d-e7ce-4790-a55d-bcfb8f76252a None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:37.936888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1337d-e7ce-4790-a55d-bcfb8f76252a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:37.937394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1337d-e7ce-4790-a55d-bcfb8f76252a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:37.938989 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a1337d-e7ce-4790-a55d-bcfb8f76252a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:37.953281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 376/769] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:37.954445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1337d-e7ce-4790-a55d-bcfb8f76252a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:37.980395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb7e97-7cd1-4abf-aefc-1f98b940d735 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:38.030305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5b518-617c-4300-890d-5605fd500a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:38.030968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5b518-617c-4300-890d-5605fd500a86 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:38.031426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5b518-617c-4300-890d-5605fd500a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:38.031704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5b518-617c-4300-890d-5605fd500a86 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:38.032480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5b518-617c-4300-890d-5605fd500a86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:38.033886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a5b518-617c-4300-890d-5605fd500a86 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:38.040986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 394/770] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Aug 19 20:55:37 2022] GET /identity/v3/services => generated 772 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:38.041833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5b518-617c-4300-890d-5605fd500a86 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:38.071906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1337d-e7ce-4790-a55d-bcfb8f76252a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:38.078713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:38.121447 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c8211-8e52-43a5-8949-e81df2f46e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:38.122503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c8211-8e52-43a5-8949-e81df2f46e33 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:38.123707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c8211-8e52-43a5-8949-e81df2f46e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:38.124341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c8211-8e52-43a5-8949-e81df2f46e33 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:38.125304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c8211-8e52-43a5-8949-e81df2f46e33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:38.125886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c8211-8e52-43a5-8949-e81df2f46e33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:38.127651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26c8211-8e52-43a5-8949-e81df2f46e33 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:38.148798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 377/771] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:38.150200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c8211-8e52-43a5-8949-e81df2f46e33 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:38.180426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5b518-617c-4300-890d-5605fd500a86 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:38.213437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:38.222218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf04df-bcd4-4434-8738-511813ce5b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:38.222861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf04df-bcd4-4434-8738-511813ce5b6f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:38.223133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf04df-bcd4-4434-8738-511813ce5b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:38.224330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf04df-bcd4-4434-8738-511813ce5b6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:38.235343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf04df-bcd4-4434-8738-511813ce5b6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:38.237750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf04df-bcd4-4434-8738-511813ce5b6f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-76cf04df-bcd4-4434-8738-511813ce5b6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:38.254548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:38.255587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:38.256454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:38.256454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:38.256454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:38.256454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:38.256454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:38.256454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:38.256454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:38.256454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:38.263827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 395/772] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:38.265173 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf04df-bcd4-4434-8738-511813ce5b6f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:38.287582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c8211-8e52-43a5-8949-e81df2f46e33 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:38.327733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b141d-7e31-4a68-b7ed-a6673f68acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:38.328900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b141d-7e31-4a68-b7ed-a6673f68acaa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:38.329432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b141d-7e31-4a68-b7ed-a6673f68acaa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:38.329953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b141d-7e31-4a68-b7ed-a6673f68acaa None admin] PATH_INFO: `/v3/services/network` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:38.330759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b141d-7e31-4a68-b7ed-a6673f68acaa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:38.332562 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b141d-7e31-4a68-b7ed-a6673f68acaa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d21b141d-7e31-4a68-b7ed-a6673f68acaa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:38.343966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:38.345007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 20:55:38.345867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:38.353022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 378/773] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:38 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:38.353022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b141d-7e31-4a68-b7ed-a6673f68acaa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:38.375259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:38.400288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf04df-bcd4-4434-8738-511813ce5b6f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:38.439883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476778d-be62-4d69-a589-6dba65c649d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:38.440916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476778d-be62-4d69-a589-6dba65c649d9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:38.441414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476778d-be62-4d69-a589-6dba65c649d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:38.441926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476778d-be62-4d69-a589-6dba65c649d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:38.443055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476778d-be62-4d69-a589-6dba65c649d9 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=61a0906b8b134c178b4dece2891f1596) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:38.443716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476778d-be62-4d69-a589-6dba65c649d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:38.445399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476778d-be62-4d69-a589-6dba65c649d9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:38.452489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 396/774] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:38 2022] GET /identity/v3/role_assignments?user.id=61a0906b8b134c178b4dece2891f1596&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=8b350845f331423cb0494e3449b58310 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:38.453709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476778d-be62-4d69-a589-6dba65c649d9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:38.476720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b141d-7e31-4a68-b7ed-a6673f68acaa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:38.499631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:38.547636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639669f-72e6-4361-a75e-b151880ec15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:38.548259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639669f-72e6-4361-a75e-b151880ec15a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:38.548537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639669f-72e6-4361-a75e-b151880ec15a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:38.549336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639669f-72e6-4361-a75e-b151880ec15a None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:38.549529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639669f-72e6-4361-a75e-b151880ec15a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:38.549872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639669f-72e6-4361-a75e-b151880ec15a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:38.551323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639669f-72e6-4361-a75e-b151880ec15a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:38.577695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 379/775] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:38.585503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476778d-be62-4d69-a589-6dba65c649d9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:38.595310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639669f-72e6-4361-a75e-b151880ec15a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:38.630007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1608d7-4934-4e0b-b8a2-fe5d56dd7351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:38.631083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1608d7-4934-4e0b-b8a2-fe5d56dd7351 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:38.631658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1608d7-4934-4e0b-b8a2-fe5d56dd7351 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:38.632143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1608d7-4934-4e0b-b8a2-fe5d56dd7351 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:38.633040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1608d7-4934-4e0b-b8a2-fe5d56dd7351 None admin] RBAC: Adding query filter params (name=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:38.633562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1608d7-4934-4e0b-b8a2-fe5d56dd7351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:38.635218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1608d7-4934-4e0b-b8a2-fe5d56dd7351 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:38.645991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 397/776] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:55:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:38.647940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1608d7-4934-4e0b-b8a2-fe5d56dd7351 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:38.663414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:38.768626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1608d7-4934-4e0b-b8a2-fe5d56dd7351 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:38.781500 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639669f-72e6-4361-a75e-b151880ec15a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:38.810151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1c170-8d51-4eda-ac6d-c917ca8cbc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:38.811329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1c170-8d51-4eda-ac6d-c917ca8cbc91 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:38.811812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1c170-8d51-4eda-ac6d-c917ca8cbc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:38.812284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1c170-8d51-4eda-ac6d-c917ca8cbc91 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:38.813199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1c170-8d51-4eda-ac6d-c917ca8cbc91 None admin] RBAC: Adding query filter params (type=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:38.813704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1c170-8d51-4eda-ac6d-c917ca8cbc91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:38.815540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1c170-8d51-4eda-ac6d-c917ca8cbc91 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:38.826279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 398/777] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:55:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:38.828789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1c170-8d51-4eda-ac6d-c917ca8cbc91 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:38.834101 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13438e9a-a473-4db6-a17a-9a8fab67f8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:38.834519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438e9a-a473-4db6-a17a-9a8fab67f8fe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:38.834793 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438e9a-a473-4db6-a17a-9a8fab67f8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:38.835062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13438e9a-a473-4db6-a17a-9a8fab67f8fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:38.835788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13438e9a-a473-4db6-a17a-9a8fab67f8fe None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=bdce5268fa7c466280dddc19869dce87) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:38.836092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13438e9a-a473-4db6-a17a-9a8fab67f8fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:38.837527 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13438e9a-a473-4db6-a17a-9a8fab67f8fe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:38.854343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 380/778] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:38 2022] GET /identity/v3/role_assignments?user.id=bdce5268fa7c466280dddc19869dce87&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:38.920394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:38.951648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1c170-8d51-4eda-ac6d-c917ca8cbc91 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:38.989632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad288d-848a-435a-8b30-fa7c17693df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:38.990324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad288d-848a-435a-8b30-fa7c17693df2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:38.990600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad288d-848a-435a-8b30-fa7c17693df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:38.990872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad288d-848a-435a-8b30-fa7c17693df2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:38.991591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad288d-848a-435a-8b30-fa7c17693df2 None admin] RBAC: Adding query filter params (interface=public, service_id=97d09698b2374d7dad8ee5ec38b5399d, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:38.991897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad288d-848a-435a-8b30-fa7c17693df2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:38.993388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad288d-848a-435a-8b30-fa7c17693df2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:39.000812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 399/779] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Aug 19 20:55:38 2022] GET /identity/v3/endpoints?service_id=97d09698b2374d7dad8ee5ec38b5399d&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:39.006682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13438e9a-a473-4db6-a17a-9a8fab67f8fe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:39.067490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:39.149290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13438e9a-a473-4db6-a17a-9a8fab67f8fe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:39.196059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560cf33d-d348-43c1-bb13-ea296caceaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:39.197014 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cf33d-d348-43c1-bb13-ea296caceaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:39.197552 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cf33d-d348-43c1-bb13-ea296caceaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:39.198060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cf33d-d348-43c1-bb13-ea296caceaa9 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:39.198897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cf33d-d348-43c1-bb13-ea296caceaa9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:39.200490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560cf33d-d348-43c1-bb13-ea296caceaa9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:39.210292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 381/780] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Aug 19 20:55:39 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:39.333654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61d9bd-7bbb-4be0-a285-bf22bc890af6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:39.334211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61d9bd-7bbb-4be0-a285-bf22bc890af6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:39.334211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61d9bd-7bbb-4be0-a285-bf22bc890af6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:39.335941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 400/781] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:39.342400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc502cc3-8bbd-4fed-8c7a-d6761960326b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:39.343020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc502cc3-8bbd-4fed-8c7a-d6761960326b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:39.343516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc502cc3-8bbd-4fed-8c7a-d6761960326b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:39.365590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dc502cc3-8bbd-4fed-8c7a-d6761960326b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:39.367242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dc502cc3-8bbd-4fed-8c7a-d6761960326b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:39.449699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 382/782] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:39.833001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61d9bd-7bbb-4be0-a285-bf22bc890af6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:40.020152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61d9bd-7bbb-4be0-a285-bf22bc890af6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:40.099219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc6829a-30d6-4eba-a4eb-2b14b719c69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:40.099913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6829a-30d6-4eba-a4eb-2b14b719c69c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:40.100192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6829a-30d6-4eba-a4eb-2b14b719c69c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:40.100466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6829a-30d6-4eba-a4eb-2b14b719c69c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:40.119810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:40.126166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc6829a-30d6-4eba-a4eb-2b14b719c69c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dcc6829a-30d6-4eba-a4eb-2b14b719c69c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:40.130942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:40.131897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:40.133356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 401/783] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:40.138677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc502cc3-8bbd-4fed-8c7a-d6761960326b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:40.162974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712b1ba-45f2-418a-89d6-07d5883b9b9a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:40.163282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712b1ba-45f2-418a-89d6-07d5883b9b9a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:40.163530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712b1ba-45f2-418a-89d6-07d5883b9b9a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:40.165129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 402/784] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:55:40.171319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b40fa-ccce-4aaa-832c-e9942cd0cf29 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:40.171624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b40fa-ccce-4aaa-832c-e9942cd0cf29 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:40.171877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b40fa-ccce-4aaa-832c-e9942cd0cf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:40.192516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ab0b40fa-ccce-4aaa-832c-e9942cd0cf29 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:40.196166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ab0b40fa-ccce-4aaa-832c-e9942cd0cf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:40.218448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:40.325454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 403/785] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:40.363237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc502cc3-8bbd-4fed-8c7a-d6761960326b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:40.443139 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e6725-50fd-4450-b23c-941fc30e1a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:40.444136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e6725-50fd-4450-b23c-941fc30e1a55 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:40.444597 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e6725-50fd-4450-b23c-941fc30e1a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:40.445055 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e6725-50fd-4450-b23c-941fc30e1a55 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:40.446132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e6725-50fd-4450-b23c-941fc30e1a55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:40.446602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e6725-50fd-4450-b23c-941fc30e1a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:40.448980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7e6725-50fd-4450-b23c-941fc30e1a55 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:40.469353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 383/786] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:40.474686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b40fa-ccce-4aaa-832c-e9942cd0cf29 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:40.619078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b40fa-ccce-4aaa-832c-e9942cd0cf29 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:40.633565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e6725-50fd-4450-b23c-941fc30e1a55 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:40.655090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:40.658467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcc7c3-b86c-46f8-b2ab-ce50eb632f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:40.663480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcc7c3-b86c-46f8-b2ab-ce50eb632f25 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:40.663480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcc7c3-b86c-46f8-b2ab-ce50eb632f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:40.663480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdcc7c3-b86c-46f8-b2ab-ce50eb632f25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:40.683654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcc7c3-b86c-46f8-b2ab-ce50eb632f25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:40.685663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdcc7c3-b86c-46f8-b2ab-ce50eb632f25 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9fdcc7c3-b86c-46f8-b2ab-ce50eb632f25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:40.712811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:40.713859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:40.714780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:40.714780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:40.714780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:40.717994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 404/787] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:40.723893 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcc7c3-b86c-46f8-b2ab-ce50eb632f25 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:40.835747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:40.852016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e6725-50fd-4450-b23c-941fc30e1a55 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:40.900214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33ab82-8d95-435c-9536-94f69c468b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:40.900773 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ab82-8d95-435c-9536-94f69c468b89 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:40.901051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ab82-8d95-435c-9536-94f69c468b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:40.901329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33ab82-8d95-435c-9536-94f69c468b89 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:40.901950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33ab82-8d95-435c-9536-94f69c468b89 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:40.903455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33ab82-8d95-435c-9536-94f69c468b89 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4d33ab82-8d95-435c-9536-94f69c468b89 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:40.913193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:40.914230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:40.915302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 384/788] 10.222.0.16 () {56 vars in 1129 bytes} [Fri Aug 19 20:55:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:40.922738 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33ab82-8d95-435c-9536-94f69c468b89 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:40.944747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdcc7c3-b86c-46f8-b2ab-ce50eb632f25 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:41.014530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc76758-7021-48ad-8331-b7b6301a0c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:41.015360 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc76758-7021-48ad-8331-b7b6301a0c51 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.015642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc76758-7021-48ad-8331-b7b6301a0c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.015913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc76758-7021-48ad-8331-b7b6301a0c51 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.016565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc76758-7021-48ad-8331-b7b6301a0c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:41.016869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc76758-7021-48ad-8331-b7b6301a0c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:41.019110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc76758-7021-48ad-8331-b7b6301a0c51 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:41.051091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 405/789] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:41.061348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc76758-7021-48ad-8331-b7b6301a0c51 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:41.119631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33ab82-8d95-435c-9536-94f69c468b89 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:41.139272 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:41.175171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32e0a7-ac09-447b-bfc7-c20807112b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:41.175849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32e0a7-ac09-447b-bfc7-c20807112b14 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.176232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32e0a7-ac09-447b-bfc7-c20807112b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.176608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32e0a7-ac09-447b-bfc7-c20807112b14 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.177463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32e0a7-ac09-447b-bfc7-c20807112b14 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:41.177943 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32e0a7-ac09-447b-bfc7-c20807112b14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:41.179546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd32e0a7-ac09-447b-bfc7-c20807112b14 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:41.189898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 385/790] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Aug 19 20:55:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:41.191386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5639cd4-098b-4155-acd4-2d62be23a5ff None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.191672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5639cd4-098b-4155-acd4-2d62be23a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.192164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5639cd4-098b-4155-acd4-2d62be23a5ff None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.197606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 386/791] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:41.197606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5639cd4-098b-4155-acd4-2d62be23a5ff None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:41.219358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc76758-7021-48ad-8331-b7b6301a0c51 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:41.223237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:41.255899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0240960-8afc-4b82-8d0a-35723df0af49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:41.256732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0240960-8afc-4b82-8d0a-35723df0af49 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.257260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0240960-8afc-4b82-8d0a-35723df0af49 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.257439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0240960-8afc-4b82-8d0a-35723df0af49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.265716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0240960-8afc-4b82-8d0a-35723df0af49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:41.267253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0240960-8afc-4b82-8d0a-35723df0af49 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e0240960-8afc-4b82-8d0a-35723df0af49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:41.280626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:41.281590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:41.282544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 406/792] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Aug 19 20:55:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:41.287726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e405723-bde6-47f9-8764-7463ca63afbc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.287726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e405723-bde6-47f9-8764-7463ca63afbc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.287726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e405723-bde6-47f9-8764-7463ca63afbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.305434 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6e405723-bde6-47f9-8764-7463ca63afbc None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:41.306990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6e405723-bde6-47f9-8764-7463ca63afbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:41.337667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5639cd4-098b-4155-acd4-2d62be23a5ff None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:41.380853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9ff7c-a9b7-4c82-9ae7-7b83096ba621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:41.381790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9ff7c-a9b7-4c82-9ae7-7b83096ba621 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.382218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9ff7c-a9b7-4c82-9ae7-7b83096ba621 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.382638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9ff7c-a9b7-4c82-9ae7-7b83096ba621 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.383449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9ff7c-a9b7-4c82-9ae7-7b83096ba621 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:41.383928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9ff7c-a9b7-4c82-9ae7-7b83096ba621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:41.385496 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9ff7c-a9b7-4c82-9ae7-7b83096ba621 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:41.392032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 387/793] 10.222.0.16 () {56 vars in 1124 bytes} [Fri Aug 19 20:55:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:41.393154 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9ff7c-a9b7-4c82-9ae7-7b83096ba621 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:41.494215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:41.505814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 407/794] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:41.505814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e405723-bde6-47f9-8764-7463ca63afbc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:41.520776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9ff7c-a9b7-4c82-9ae7-7b83096ba621 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:41.548920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:41.569825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc5a5b-c4ca-46ef-9977-9268c72a3430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:41.570976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc5a5b-c4ca-46ef-9977-9268c72a3430 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.571900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc5a5b-c4ca-46ef-9977-9268c72a3430 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.572326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc5a5b-c4ca-46ef-9977-9268c72a3430 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.573482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc5a5b-c4ca-46ef-9977-9268c72a3430 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:41.574224 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc5a5b-c4ca-46ef-9977-9268c72a3430 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:41.576867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc5a5b-c4ca-46ef-9977-9268c72a3430 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:41.608364 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 388/795] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Aug 19 20:55:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:41.608364 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d755f-12ab-4945-bca4-3b5399272175 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.608364 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d755f-12ab-4945-bca4-3b5399272175 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.608364 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d755f-12ab-4945-bca4-3b5399272175 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.611644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 389/796] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:41.612121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d755f-12ab-4945-bca4-3b5399272175 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:41.641921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e405723-bde6-47f9-8764-7463ca63afbc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:41.680604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98527277-7609-481e-8310-73ca0ded5285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:41.681259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98527277-7609-481e-8310-73ca0ded5285 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.681539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98527277-7609-481e-8310-73ca0ded5285 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.681901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98527277-7609-481e-8310-73ca0ded5285 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.682624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98527277-7609-481e-8310-73ca0ded5285 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:41.684381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98527277-7609-481e-8310-73ca0ded5285 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:41.713505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 408/797] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 20:55:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:41.715388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbf2c9-8658-4e01-886f-c5801d6b4af7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.715751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbf2c9-8658-4e01-886f-c5801d6b4af7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.716068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbf2c9-8658-4e01-886f-c5801d6b4af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.742073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a5cbf2c9-8658-4e01-886f-c5801d6b4af7 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:41.743277 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a5cbf2c9-8658-4e01-886f-c5801d6b4af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:41.753264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d755f-12ab-4945-bca4-3b5399272175 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:41.797723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e13852-71c7-4dcc-ba44-8740c11070ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:41.798393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e13852-71c7-4dcc-ba44-8740c11070ae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.798674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e13852-71c7-4dcc-ba44-8740c11070ae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.798949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e13852-71c7-4dcc-ba44-8740c11070ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.808977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:41.829417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 409/798] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:41.830794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41942312-2642-4c7e-bfc0-b553b054ade0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.831272 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41942312-2642-4c7e-bfc0-b553b054ade0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.831618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41942312-2642-4c7e-bfc0-b553b054ade0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.837309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 410/799] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:41.838852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa6ea2-c384-4176-b4ff-e2e2708bc33b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:41.839186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa6ea2-c384-4176-b4ff-e2e2708bc33b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:41.839469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa6ea2-c384-4176-b4ff-e2e2708bc33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:41.860200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-49fa6ea2-c384-4176-b4ff-e2e2708bc33b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:41.861437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-49fa6ea2-c384-4176-b4ff-e2e2708bc33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:41.875553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:41.887505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e13852-71c7-4dcc-ba44-8740c11070ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:41.890061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e13852-71c7-4dcc-ba44-8740c11070ae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:41.945945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 411/800] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:55:41.946824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa6ea2-c384-4176-b4ff-e2e2708bc33b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.028169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 390/801] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:55:42.036528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e13852-71c7-4dcc-ba44-8740c11070ae None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.077473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa6ea2-c384-4176-b4ff-e2e2708bc33b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:42.091013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:42.122190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386717a2-fe2a-4641-b316-7d0a054c04f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:42.123357 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386717a2-fe2a-4641-b316-7d0a054c04f3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:42.123642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386717a2-fe2a-4641-b316-7d0a054c04f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:42.123898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386717a2-fe2a-4641-b316-7d0a054c04f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:42.134969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386717a2-fe2a-4641-b316-7d0a054c04f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-386717a2-fe2a-4641-b316-7d0a054c04f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:42.140091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:42.141041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:42.142084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 412/802] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:42.150328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386717a2-fe2a-4641-b316-7d0a054c04f3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.161086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:42.171322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e13852-71c7-4dcc-ba44-8740c11070ae None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:42.222575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6328d-9c27-4a88-8b7f-1458141d97c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:42.223537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6328d-9c27-4a88-8b7f-1458141d97c4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:42.223821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6328d-9c27-4a88-8b7f-1458141d97c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:42.224544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f6328d-9c27-4a88-8b7f-1458141d97c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:42.243066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6328d-9c27-4a88-8b7f-1458141d97c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:42.245403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f6328d-9c27-4a88-8b7f-1458141d97c4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-64f6328d-9c27-4a88-8b7f-1458141d97c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:42.265214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:42.266421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:42.267380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:42.267380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:42.267380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:42.267567 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386717a2-fe2a-4641-b316-7d0a054c04f3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:42.267644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 391/803] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:42.267644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6328d-9c27-4a88-8b7f-1458141d97c4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.310007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01da36-0068-4039-8436-01ecb5a43246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:42.310819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01da36-0068-4039-8436-01ecb5a43246 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:42.311009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01da36-0068-4039-8436-01ecb5a43246 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:42.311252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee01da36-0068-4039-8436-01ecb5a43246 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:42.311930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01da36-0068-4039-8436-01ecb5a43246 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:42.312236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01da36-0068-4039-8436-01ecb5a43246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:42.313719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee01da36-0068-4039-8436-01ecb5a43246 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:42.327465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 413/804] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:42.327465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01da36-0068-4039-8436-01ecb5a43246 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.374047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:42.389864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f6328d-9c27-4a88-8b7f-1458141d97c4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:42.402879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:42.444806 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee01da36-0068-4039-8436-01ecb5a43246 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:42.445350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc300430-0270-4198-9bc2-399b565fc437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:42.446027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc300430-0270-4198-9bc2-399b565fc437 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:42.446307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc300430-0270-4198-9bc2-399b565fc437 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:42.446579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc300430-0270-4198-9bc2-399b565fc437 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:42.458495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc300430-0270-4198-9bc2-399b565fc437 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-dc300430-0270-4198-9bc2-399b565fc437 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:42.464646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:42.465612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 392/805] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:42.466592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc300430-0270-4198-9bc2-399b565fc437 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.495874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c1273-5c63-4fe6-af2b-38d48807d666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:42.496500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c1273-5c63-4fe6-af2b-38d48807d666 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:42.496776 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c1273-5c63-4fe6-af2b-38d48807d666 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:42.497044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c1273-5c63-4fe6-af2b-38d48807d666 None admin] PATH_INFO: `/v3/services/network` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:42.498308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c1273-5c63-4fe6-af2b-38d48807d666 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:42.500039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c1273-5c63-4fe6-af2b-38d48807d666 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c05c1273-5c63-4fe6-af2b-38d48807d666 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:42.509050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:42.514455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:42.515511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 414/806] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:42 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:42.516324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c1273-5c63-4fe6-af2b-38d48807d666 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.587586 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc300430-0270-4198-9bc2-399b565fc437 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:42.621790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416be84-5cf4-42c9-a0cc-668e51dd74e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:42.622514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416be84-5cf4-42c9-a0cc-668e51dd74e1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:42.623427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416be84-5cf4-42c9-a0cc-668e51dd74e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:42.623919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416be84-5cf4-42c9-a0cc-668e51dd74e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:42.624643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416be84-5cf4-42c9-a0cc-668e51dd74e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:42.624945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416be84-5cf4-42c9-a0cc-668e51dd74e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:42.626350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416be84-5cf4-42c9-a0cc-668e51dd74e1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:42.627358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c1273-5c63-4fe6-af2b-38d48807d666 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:42.643621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 393/807] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:42.643621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416be84-5cf4-42c9-a0cc-668e51dd74e1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.656532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:42.664950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:42.672483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f9885-62c4-40d9-b967-c95d1fc2d27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:42.673139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9885-62c4-40d9-b967-c95d1fc2d27e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:42.673649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9885-62c4-40d9-b967-c95d1fc2d27e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:42.673978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f9885-62c4-40d9-b967-c95d1fc2d27e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:42.685654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f9885-62c4-40d9-b967-c95d1fc2d27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:42.687323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f9885-62c4-40d9-b967-c95d1fc2d27e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-797f9885-62c4-40d9-b967-c95d1fc2d27e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:42.703507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:42.704469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:42.705339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:42.705339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:42.705339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:42.705339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 415/808] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:42.705605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f9885-62c4-40d9-b967-c95d1fc2d27e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.769026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416be84-5cf4-42c9-a0cc-668e51dd74e1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:42.807620 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053a92d-7ee7-4eb1-b353-707b2690b38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:42.808236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053a92d-7ee7-4eb1-b353-707b2690b38a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:42.808522 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053a92d-7ee7-4eb1-b353-707b2690b38a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:42.808798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053a92d-7ee7-4eb1-b353-707b2690b38a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:42.809537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053a92d-7ee7-4eb1-b353-707b2690b38a None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:42.809892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053a92d-7ee7-4eb1-b353-707b2690b38a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:42.811284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053a92d-7ee7-4eb1-b353-707b2690b38a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:42.819600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 394/809] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:42.824192 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053a92d-7ee7-4eb1-b353-707b2690b38a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.834605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f9885-62c4-40d9-b967-c95d1fc2d27e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:42.875600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573644d5-677f-4484-b20f-02808250e246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:42.876134 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573644d5-677f-4484-b20f-02808250e246 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:42.876415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573644d5-677f-4484-b20f-02808250e246 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:42.876707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573644d5-677f-4484-b20f-02808250e246 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:42.877320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573644d5-677f-4484-b20f-02808250e246 None admin] RBAC: Adding query filter params (name=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:42.877819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573644d5-677f-4484-b20f-02808250e246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:42.879042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573644d5-677f-4484-b20f-02808250e246 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:42.889337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 416/810] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:55:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:42.890325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573644d5-677f-4484-b20f-02808250e246 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:42.900939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:42.938991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053a92d-7ee7-4eb1-b353-707b2690b38a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:42.943919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:42.990317 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba108d9-8984-4306-91bb-272ea55d0541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:42.991053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba108d9-8984-4306-91bb-272ea55d0541 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:42.991339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba108d9-8984-4306-91bb-272ea55d0541 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:42.991612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba108d9-8984-4306-91bb-272ea55d0541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:43.000141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba108d9-8984-4306-91bb-272ea55d0541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:43.001601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba108d9-8984-4306-91bb-272ea55d0541 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:43.002579 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573644d5-677f-4484-b20f-02808250e246 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2ba108d9-8984-4306-91bb-272ea55d0541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:43.015448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:43.016521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:43.017477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:43.017477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:43.017477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:43.017477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:43.017477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:43.017477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:43.017477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:43.017477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:43.017477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 395/811] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:43.017994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba108d9-8984-4306-91bb-272ea55d0541 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:43.045289 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83ea23-36c9-4c52-b2bd-8217b02fe509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:43.045945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83ea23-36c9-4c52-b2bd-8217b02fe509 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:43.046357 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83ea23-36c9-4c52-b2bd-8217b02fe509 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:43.046646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83ea23-36c9-4c52-b2bd-8217b02fe509 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:43.048527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83ea23-36c9-4c52-b2bd-8217b02fe509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:43.049054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83ea23-36c9-4c52-b2bd-8217b02fe509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:43.051237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83ea23-36c9-4c52-b2bd-8217b02fe509 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:43.071065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 417/812] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:43.071065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83ea23-36c9-4c52-b2bd-8217b02fe509 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:43.157376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba108d9-8984-4306-91bb-272ea55d0541 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:43.190048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83ea23-36c9-4c52-b2bd-8217b02fe509 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:43.190294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:43.197125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73709295-46b5-4998-95de-670ef9157ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:43.197795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73709295-46b5-4998-95de-670ef9157ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:43.198069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73709295-46b5-4998-95de-670ef9157ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:43.198343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73709295-46b5-4998-95de-670ef9157ac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:43.202032 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:43.212276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73709295-46b5-4998-95de-670ef9157ac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:43.213754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73709295-46b5-4998-95de-670ef9157ac3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-73709295-46b5-4998-95de-670ef9157ac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:43.230066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:43.232286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:43.233257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:43.233257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:43.233257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:43.233257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 396/813] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:43.233257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73709295-46b5-4998-95de-670ef9157ac3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:43.239418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58f529-45a6-4c1d-9734-5fd00d473844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:43.240093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58f529-45a6-4c1d-9734-5fd00d473844 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:43.240372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58f529-45a6-4c1d-9734-5fd00d473844 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:43.240619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58f529-45a6-4c1d-9734-5fd00d473844 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:43.241319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58f529-45a6-4c1d-9734-5fd00d473844 None admin] RBAC: Adding query filter params (type=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:43.241673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58f529-45a6-4c1d-9734-5fd00d473844 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:43.243100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58f529-45a6-4c1d-9734-5fd00d473844 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:43.255580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 418/814] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:55:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:43.255580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58f529-45a6-4c1d-9734-5fd00d473844 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:43.353569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73709295-46b5-4998-95de-670ef9157ac3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:43.373649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58f529-45a6-4c1d-9734-5fd00d473844 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:43.392702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2a96f-1f40-4858-b767-0b7dbee17700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:43.393419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2a96f-1f40-4858-b767-0b7dbee17700 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:43.393784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2a96f-1f40-4858-b767-0b7dbee17700 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:43.394134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2a96f-1f40-4858-b767-0b7dbee17700 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:43.394909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2a96f-1f40-4858-b767-0b7dbee17700 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:43.395285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2a96f-1f40-4858-b767-0b7dbee17700 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:43.396804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2a96f-1f40-4858-b767-0b7dbee17700 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:43.416788 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d7a8a-ed7d-434b-9f97-5adb6d7a7b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:43.416981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 397/815] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:43.417075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d7a8a-ed7d-434b-9f97-5adb6d7a7b37 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:43.417075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d7a8a-ed7d-434b-9f97-5adb6d7a7b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:43.417240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2a96f-1f40-4858-b767-0b7dbee17700 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:43.417323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d7a8a-ed7d-434b-9f97-5adb6d7a7b37 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:43.432683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d7a8a-ed7d-434b-9f97-5adb6d7a7b37 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:43.434476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d7a8a-ed7d-434b-9f97-5adb6d7a7b37 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-029d7a8a-ed7d-434b-9f97-5adb6d7a7b37 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:43.448934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:43.450399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:43.451903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:43.451903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:43.451903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:43.451903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:43.451903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:43.451903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:43.451903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:43.451903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:43.451903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:43.451903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:43.452889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 419/816] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:43.453750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d7a8a-ed7d-434b-9f97-5adb6d7a7b37 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:43.483934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:43.489668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:43.555280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2a96f-1f40-4858-b767-0b7dbee17700 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:43.574712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d7a8a-ed7d-434b-9f97-5adb6d7a7b37 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:43.602200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fecf2-1502-415c-8260-bcab8282a08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:43.602995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fecf2-1502-415c-8260-bcab8282a08d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:43.603278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fecf2-1502-415c-8260-bcab8282a08d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:43.603547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fecf2-1502-415c-8260-bcab8282a08d None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:43.604201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fecf2-1502-415c-8260-bcab8282a08d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:43.604502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fecf2-1502-415c-8260-bcab8282a08d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:43.605905 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fecf2-1502-415c-8260-bcab8282a08d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:43.616207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15f5dd-2e60-4435-9fe0-e6e4bfe18694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:43.616751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15f5dd-2e60-4435-9fe0-e6e4bfe18694 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:43.617032 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15f5dd-2e60-4435-9fe0-e6e4bfe18694 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:43.617306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15f5dd-2e60-4435-9fe0-e6e4bfe18694 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:43.618036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15f5dd-2e60-4435-9fe0-e6e4bfe18694 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:43.619734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15f5dd-2e60-4435-9fe0-e6e4bfe18694 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:43.624977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 398/817] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:43.624977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fecf2-1502-415c-8260-bcab8282a08d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:43.646306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 420/818] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 20:55:43 2022] POST /identity/v3/endpoints => generated 337 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:43.659304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15f5dd-2e60-4435-9fe0-e6e4bfe18694 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:43.780618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fecf2-1502-415c-8260-bcab8282a08d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:43.784635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:43.786969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:43.824252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15f5dd-2e60-4435-9fe0-e6e4bfe18694 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:43.854584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc499c-b02f-4ccf-84a0-d35d113763e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:43.855397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc499c-b02f-4ccf-84a0-d35d113763e8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:43.855680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc499c-b02f-4ccf-84a0-d35d113763e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:43.856120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc499c-b02f-4ccf-84a0-d35d113763e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:43.857832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc499c-b02f-4ccf-84a0-d35d113763e8 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=43a9ad5341b54c40a5427045660363c3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:43.858121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc499c-b02f-4ccf-84a0-d35d113763e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:43.859919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cc499c-b02f-4ccf-84a0-d35d113763e8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:43.867676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 399/819] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:43 2022] GET /identity/v3/role_assignments?user.id=43a9ad5341b54c40a5427045660363c3&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=8b350845f331423cb0494e3449b58310 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:43.868347 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc499c-b02f-4ccf-84a0-d35d113763e8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:43.870518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd47b87-d31b-4f75-bc35-92f0baf3f3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:43.871222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd47b87-d31b-4f75-bc35-92f0baf3f3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:43.871401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd47b87-d31b-4f75-bc35-92f0baf3f3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:43.871667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd47b87-d31b-4f75-bc35-92f0baf3f3d7 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:43.874537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd47b87-d31b-4f75-bc35-92f0baf3f3d7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:43.874537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd47b87-d31b-4f75-bc35-92f0baf3f3d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:43.875664 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd47b87-d31b-4f75-bc35-92f0baf3f3d7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:43.897418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 421/820] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Aug 19 20:55:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:43.903017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd47b87-d31b-4f75-bc35-92f0baf3f3d7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:43.997009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc499c-b02f-4ccf-84a0-d35d113763e8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.039297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3b6c8-f943-4b5a-b13d-90c8fb161cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.039864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3b6c8-f943-4b5a-b13d-90c8fb161cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.040144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3b6c8-f943-4b5a-b13d-90c8fb161cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.040413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3b6c8-f943-4b5a-b13d-90c8fb161cf7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.049672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3b6c8-f943-4b5a-b13d-90c8fb161cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.051231 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3b6c8-f943-4b5a-b13d-90c8fb161cf7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.051884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd47b87-d31b-4f75-bc35-92f0baf3f3d7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-03f3b6c8-f943-4b5a-b13d-90c8fb161cf7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:44.064245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:44.065218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 400/821] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Aug 19 20:55:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119c092-7f3d-4cd6-89a9-cad37fd7bcd1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.066294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119c092-7f3d-4cd6-89a9-cad37fd7bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.067008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119c092-7f3d-4cd6-89a9-cad37fd7bcd1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.068058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 401/822] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:44.074846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119c092-7f3d-4cd6-89a9-cad37fd7bcd1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:44.108579 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1a79b9-91fd-40f1-bb04-89514827db8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.109111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:44.109214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a79b9-91fd-40f1-bb04-89514827db8c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.109483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a79b9-91fd-40f1-bb04-89514827db8c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.109757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1a79b9-91fd-40f1-bb04-89514827db8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.121326 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a79b9-91fd-40f1-bb04-89514827db8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.123653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1a79b9-91fd-40f1-bb04-89514827db8c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.125623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ae1a79b9-91fd-40f1-bb04-89514827db8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:44.139875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:44.140895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:44.141845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:44.141845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:44.141845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:44.141845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 422/823] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:44.142537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b4007-5e29-4895-b233-79894768bc76 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.143189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b4007-5e29-4895-b233-79894768bc76 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.143189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b4007-5e29-4895-b233-79894768bc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.163234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-773b4007-5e29-4895-b233-79894768bc76 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:44.164717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-773b4007-5e29-4895-b233-79894768bc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:44.201412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119c092-7f3d-4cd6-89a9-cad37fd7bcd1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.234620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 423/824] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:44.239126 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0a51-fbde-4c27-aed8-da6327fe7f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.239870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0a51-fbde-4c27-aed8-da6327fe7f12 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.240209 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0a51-fbde-4c27-aed8-da6327fe7f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.240545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0a51-fbde-4c27-aed8-da6327fe7f12 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.241426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b0a51-fbde-4c27-aed8-da6327fe7f12 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:44.241927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b0a51-fbde-4c27-aed8-da6327fe7f12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.243006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b4007-5e29-4895-b233-79894768bc76 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:44.244167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b0a51-fbde-4c27-aed8-da6327fe7f12 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.271408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 402/825] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Aug 19 20:55:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:44.273425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0a51-fbde-4c27-aed8-da6327fe7f12 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:44.361646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b4007-5e29-4895-b233-79894768bc76 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.366294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:44.383926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0a51-fbde-4c27-aed8-da6327fe7f12 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.390713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:44.403074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0c24f-ed2a-4507-a135-c7103be0a7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.404433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0c24f-ed2a-4507-a135-c7103be0a7b2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.404958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0c24f-ed2a-4507-a135-c7103be0a7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.405321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0c24f-ed2a-4507-a135-c7103be0a7b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.406129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0c24f-ed2a-4507-a135-c7103be0a7b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:44.406528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0c24f-ed2a-4507-a135-c7103be0a7b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.407989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a0c24f-ed2a-4507-a135-c7103be0a7b2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.421605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 424/826] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:44.424113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4c56-f107-46fc-8245-88750ccd0b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.424752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4c56-f107-46fc-8245-88750ccd0b32 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.425042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4c56-f107-46fc-8245-88750ccd0b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.425302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4c56-f107-46fc-8245-88750ccd0b32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.427214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0c24f-ed2a-4507-a135-c7103be0a7b2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:44.436520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4c56-f107-46fc-8245-88750ccd0b32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.438036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4c56-f107-46fc-8245-88750ccd0b32 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d89d4c56-f107-46fc-8245-88750ccd0b32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:44.454595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:44.455540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:44.456400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:44.456400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:44.456400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:44.460539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 403/827] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:44.461686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4c56-f107-46fc-8245-88750ccd0b32 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:44.541804 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a0c24f-ed2a-4507-a135-c7103be0a7b2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.579358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52d5a5-70a7-4090-af1d-6558c2ed50f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.579977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4c56-f107-46fc-8245-88750ccd0b32 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.580096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52d5a5-70a7-4090-af1d-6558c2ed50f2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.580344 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52d5a5-70a7-4090-af1d-6558c2ed50f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.580555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d52d5a5-70a7-4090-af1d-6558c2ed50f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.587822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52d5a5-70a7-4090-af1d-6558c2ed50f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.589574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d52d5a5-70a7-4090-af1d-6558c2ed50f2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2d52d5a5-70a7-4090-af1d-6558c2ed50f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:44.602287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:44.603412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:44.604743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:44.604743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:44.604743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:44.604743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:44.604743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:44.604743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:44.604743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:44.604743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:44.608520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 425/828] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:44.609444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52d5a5-70a7-4090-af1d-6558c2ed50f2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:44.625198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdc952-2ed8-4f76-9101-61a0b92862ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.626240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdc952-2ed8-4f76-9101-61a0b92862ee None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.626528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdc952-2ed8-4f76-9101-61a0b92862ee None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.626970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdc952-2ed8-4f76-9101-61a0b92862ee None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.627638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdc952-2ed8-4f76-9101-61a0b92862ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:44.627945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdc952-2ed8-4f76-9101-61a0b92862ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.629355 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fdc952-2ed8-4f76-9101-61a0b92862ee None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.638557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:44.649754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 404/829] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:44.650554 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdc952-2ed8-4f76-9101-61a0b92862ee None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:44.671294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:44.736385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d52d5a5-70a7-4090-af1d-6558c2ed50f2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.760185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fdc952-2ed8-4f76-9101-61a0b92862ee None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.774990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be998fc2-db98-41ca-8272-6ef0d1154b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.775854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be998fc2-db98-41ca-8272-6ef0d1154b5f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.776291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be998fc2-db98-41ca-8272-6ef0d1154b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.776795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be998fc2-db98-41ca-8272-6ef0d1154b5f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.777818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be998fc2-db98-41ca-8272-6ef0d1154b5f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.779288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be998fc2-db98-41ca-8272-6ef0d1154b5f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-be998fc2-db98-41ca-8272-6ef0d1154b5f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:44.788520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:44.789482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:44.790425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 426/830] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:55:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:44.791207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be998fc2-db98-41ca-8272-6ef0d1154b5f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:44.802597 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f707ca9-ba5d-4404-8d7f-b80ea1be0cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.803171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f707ca9-ba5d-4404-8d7f-b80ea1be0cdf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.803672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f707ca9-ba5d-4404-8d7f-b80ea1be0cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.803856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f707ca9-ba5d-4404-8d7f-b80ea1be0cdf None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.804691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f707ca9-ba5d-4404-8d7f-b80ea1be0cdf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:44.805095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f707ca9-ba5d-4404-8d7f-b80ea1be0cdf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.806693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f707ca9-ba5d-4404-8d7f-b80ea1be0cdf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.818788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 405/831] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:44.819525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f707ca9-ba5d-4404-8d7f-b80ea1be0cdf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:44.880065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:44.911836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be998fc2-db98-41ca-8272-6ef0d1154b5f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.921239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:44.925005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f707ca9-ba5d-4404-8d7f-b80ea1be0cdf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:44.953833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e65e9a-0686-47fd-9eef-c1471721c95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.954465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e65e9a-0686-47fd-9eef-c1471721c95f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.954744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e65e9a-0686-47fd-9eef-c1471721c95f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.955019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e65e9a-0686-47fd-9eef-c1471721c95f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.961070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee0e1b-2fe1-4e11-8302-be170375e86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:44.961687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee0e1b-2fe1-4e11-8302-be170375e86e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:44.961997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee0e1b-2fe1-4e11-8302-be170375e86e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:44.962283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e65e9a-0686-47fd-9eef-c1471721c95f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.962387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee0e1b-2fe1-4e11-8302-be170375e86e None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:44.962899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee0e1b-2fe1-4e11-8302-be170375e86e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:44.963201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee0e1b-2fe1-4e11-8302-be170375e86e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:44.963708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e65e9a-0686-47fd-9eef-c1471721c95f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.964555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee0e1b-2fe1-4e11-8302-be170375e86e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:44.974278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 406/832] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:55:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:44.974278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee0e1b-2fe1-4e11-8302-be170375e86e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e4e65e9a-0686-47fd-9eef-c1471721c95f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:44.980196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:44.981089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:44.982249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:44.982249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:44.982249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:44.982249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:44.982249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:44.982249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:44.982249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:44.982249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:44.986075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 427/833] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:44.987024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e65e9a-0686-47fd-9eef-c1471721c95f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:45.091327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee0e1b-2fe1-4e11-8302-be170375e86e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:45.101052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e65e9a-0686-47fd-9eef-c1471721c95f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:45.117015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:45.131886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521018e-e52b-47d1-a369-fe966b4e2c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:45.132848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521018e-e52b-47d1-a369-fe966b4e2c4d None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:45.133277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521018e-e52b-47d1-a369-fe966b4e2c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:45.133686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521018e-e52b-47d1-a369-fe966b4e2c4d None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/61a0906b8b134c178b4dece2891f1596/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:45.135615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2e85b-4f8c-4ec0-af60-fed71668a58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:45.136134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521018e-e52b-47d1-a369-fe966b4e2c4d None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=61a0906b8b134c178b4dece2891f1596, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:45.136265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2e85b-4f8c-4ec0-af60-fed71668a58b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:45.136483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2e85b-4f8c-4ec0-af60-fed71668a58b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:45.136752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2e85b-4f8c-4ec0-af60-fed71668a58b None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:45.137379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2e85b-4f8c-4ec0-af60-fed71668a58b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:45.137680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2e85b-4f8c-4ec0-af60-fed71668a58b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:45.139378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521018e-e52b-47d1-a369-fe966b4e2c4d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:45.139610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2e85b-4f8c-4ec0-af60-fed71668a58b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:45.144125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 428/834] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:55:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:45.145112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2e85b-4f8c-4ec0-af60-fed71668a58b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:45.157259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 407/835] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:55:44 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/61a0906b8b134c178b4dece2891f1596/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:55:45.205154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:45.297498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2e85b-4f8c-4ec0-af60-fed71668a58b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:45.339100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4f263-0799-42b8-9d60-dabd5a4b715d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:45.339676 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4f263-0799-42b8-9d60-dabd5a4b715d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:45.339949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4f263-0799-42b8-9d60-dabd5a4b715d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:45.340222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4f263-0799-42b8-9d60-dabd5a4b715d None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:45.340918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4f263-0799-42b8-9d60-dabd5a4b715d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:45.341494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4f263-0799-42b8-9d60-dabd5a4b715d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:45.343063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4f263-0799-42b8-9d60-dabd5a4b715d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:45.356208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 429/836] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:45.362969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521018e-e52b-47d1-a369-fe966b4e2c4d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:45.470906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521018e-e52b-47d1-a369-fe966b4e2c4d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:45.506788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e752a-ef64-410b-8413-33be514ad96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:45.507463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e752a-ef64-410b-8413-33be514ad96d None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:45.507740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e752a-ef64-410b-8413-33be514ad96d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:45.508013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e752a-ef64-410b-8413-33be514ad96d None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/bdce5268fa7c466280dddc19869dce87/roles/a797bb15d18b42148f36a77bd6d59606` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:45.525076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e752a-ef64-410b-8413-33be514ad96d None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=bdce5268fa7c466280dddc19869dce87, role_id=a797bb15d18b42148f36a77bd6d59606)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:45.526551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e752a-ef64-410b-8413-33be514ad96d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:45.544061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 408/837] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:55:45 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/bdce5268fa7c466280dddc19869dce87/roles/a797bb15d18b42148f36a77bd6d59606 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:55:45.949518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36347f8e-7c3b-4507-bc6f-e419a14bb40d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:45.950089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36347f8e-7c3b-4507-bc6f-e419a14bb40d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:45.950089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36347f8e-7c3b-4507-bc6f-e419a14bb40d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:45.952267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 430/838] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:45.957705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afe035-e36b-4382-8307-cfa6659cb50c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:45.958027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afe035-e36b-4382-8307-cfa6659cb50c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:45.958308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afe035-e36b-4382-8307-cfa6659cb50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:45.977013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-56afe035-e36b-4382-8307-cfa6659cb50c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:45.978264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-56afe035-e36b-4382-8307-cfa6659cb50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:46.011143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:46.058360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 409/839] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:46.349088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36347f8e-7c3b-4507-bc6f-e419a14bb40d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:46.454721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:46.464539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36347f8e-7c3b-4507-bc6f-e419a14bb40d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:46.501101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16845f2a-5dc8-4cf0-86dd-da8cb9c919d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:46.501732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16845f2a-5dc8-4cf0-86dd-da8cb9c919d1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:46.502036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16845f2a-5dc8-4cf0-86dd-da8cb9c919d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:46.502304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16845f2a-5dc8-4cf0-86dd-da8cb9c919d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:46.502887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16845f2a-5dc8-4cf0-86dd-da8cb9c919d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:46.504261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16845f2a-5dc8-4cf0-86dd-da8cb9c919d1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-16845f2a-5dc8-4cf0-86dd-da8cb9c919d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:46.513818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:46.514754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:46.515620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 431/840] 10.222.0.16 () {56 vars in 1132 bytes} [Fri Aug 19 20:55:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:46.521449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afe035-e36b-4382-8307-cfa6659cb50c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:46.640107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afe035-e36b-4382-8307-cfa6659cb50c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:46.682287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dd5642-01b9-4448-a727-7169bdb34515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:46.682966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd5642-01b9-4448-a727-7169bdb34515 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:46.683242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd5642-01b9-4448-a727-7169bdb34515 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:46.683513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd5642-01b9-4448-a727-7169bdb34515 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:46.684217 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd5642-01b9-4448-a727-7169bdb34515 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:46.684559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd5642-01b9-4448-a727-7169bdb34515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:46.686709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dd5642-01b9-4448-a727-7169bdb34515 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:46.692043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 410/841] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:55:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:46.699081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16845f2a-5dc8-4cf0-86dd-da8cb9c919d1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:46.828449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16845f2a-5dc8-4cf0-86dd-da8cb9c919d1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:46.864467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291c878-00c6-4d84-b00a-b12cf659a824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:46.865082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291c878-00c6-4d84-b00a-b12cf659a824 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:46.865358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291c878-00c6-4d84-b00a-b12cf659a824 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:46.865623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291c878-00c6-4d84-b00a-b12cf659a824 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:46.866286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291c878-00c6-4d84-b00a-b12cf659a824 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:46.866583 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291c878-00c6-4d84-b00a-b12cf659a824 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:46.867939 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291c878-00c6-4d84-b00a-b12cf659a824 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:46.873650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 432/842] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:55:46 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:47.177822 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5d095-9c7f-4479-b14d-e2c62bd5e0fb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:47.178171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5d095-9c7f-4479-b14d-e2c62bd5e0fb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:47.178382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5d095-9c7f-4479-b14d-e2c62bd5e0fb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:47.179987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 411/843] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:47.185814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756ded6-15e9-4c64-bf36-f01e6835d1bd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:47.186310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756ded6-15e9-4c64-bf36-f01e6835d1bd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:47.186387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a756ded6-15e9-4c64-bf36-f01e6835d1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:47.205923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a756ded6-15e9-4c64-bf36-f01e6835d1bd None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:47.207147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a756ded6-15e9-4c64-bf36-f01e6835d1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:47.225743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:47.288945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 433/844] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:47.327385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f38e3-377a-4611-962f-d9ff11893131 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:47.327723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f38e3-377a-4611-962f-d9ff11893131 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:47.327942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f38e3-377a-4611-962f-d9ff11893131 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:47.329616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 412/845] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:47.335774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19910827-5c3f-47d8-a62e-2a8a328a79ec None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:47.336104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19910827-5c3f-47d8-a62e-2a8a328a79ec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:47.336331 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19910827-5c3f-47d8-a62e-2a8a328a79ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:47.361955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-19910827-5c3f-47d8-a62e-2a8a328a79ec None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:47.363164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-19910827-5c3f-47d8-a62e-2a8a328a79ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:47.438936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 434/846] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:47.571334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f38e3-377a-4611-962f-d9ff11893131 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:47.600155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:47.617361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c9006-955a-48eb-852d-f61ed47419db None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:47.617965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c9006-955a-48eb-852d-f61ed47419db None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:47.618247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c9006-955a-48eb-852d-f61ed47419db None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:47.620723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 435/847] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:47.625936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680426e9-5e09-4b0c-80d2-95ddae86e894 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:47.626210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680426e9-5e09-4b0c-80d2-95ddae86e894 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:47.626489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680426e9-5e09-4b0c-80d2-95ddae86e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:47.646545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-680426e9-5e09-4b0c-80d2-95ddae86e894 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:47.647771 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-680426e9-5e09-4b0c-80d2-95ddae86e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:47.687884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f38e3-377a-4611-962f-d9ff11893131 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:47.720977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 436/848] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:47.734117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa86ef-c816-4cfb-bc50-af3422e82d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:47.736473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa86ef-c816-4cfb-bc50-af3422e82d78 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:47.736473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa86ef-c816-4cfb-bc50-af3422e82d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:47.736473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa86ef-c816-4cfb-bc50-af3422e82d78 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:47.736722 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680426e9-5e09-4b0c-80d2-95ddae86e894 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:47.736827 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa86ef-c816-4cfb-bc50-af3422e82d78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:47.737547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa86ef-c816-4cfb-bc50-af3422e82d78 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:47.758176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:47.758811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 413/849] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:55:47 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:47.868397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680426e9-5e09-4b0c-80d2-95ddae86e894 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:47.917342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed2b2a-4571-4df0-8be1-84c1f72e4988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:47.918127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed2b2a-4571-4df0-8be1-84c1f72e4988 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:47.918412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed2b2a-4571-4df0-8be1-84c1f72e4988 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:47.918687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed2b2a-4571-4df0-8be1-84c1f72e4988 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:47.930801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed2b2a-4571-4df0-8be1-84c1f72e4988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-79ed2b2a-4571-4df0-8be1-84c1f72e4988 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:47.936468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:47.937586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:47.938768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 437/850] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:47.944431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa86ef-c816-4cfb-bc50-af3422e82d78 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.004473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed2b2a-4571-4df0-8be1-84c1f72e4988 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.058790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:48.080540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa86ef-c816-4cfb-bc50-af3422e82d78 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:48.106449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:48.125800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a51a73-a3b7-49fb-b145-f4da2fb1293b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:48.126409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a51a73-a3b7-49fb-b145-f4da2fb1293b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:48.126916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a51a73-a3b7-49fb-b145-f4da2fb1293b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:48.127042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a51a73-a3b7-49fb-b145-f4da2fb1293b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:48.127681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a51a73-a3b7-49fb-b145-f4da2fb1293b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:48.127984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a51a73-a3b7-49fb-b145-f4da2fb1293b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:48.129357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a51a73-a3b7-49fb-b145-f4da2fb1293b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:48.143763 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 414/851] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:55:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:48.144977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a51a73-a3b7-49fb-b145-f4da2fb1293b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.153160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed2b2a-4571-4df0-8be1-84c1f72e4988 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:48.199866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6884-1715-4d93-9f2a-a9600932c161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:48.201933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6884-1715-4d93-9f2a-a9600932c161 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:48.202359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6884-1715-4d93-9f2a-a9600932c161 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:48.203090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6884-1715-4d93-9f2a-a9600932c161 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:48.217125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c6884-1715-4d93-9f2a-a9600932c161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4a1c6884-1715-4d93-9f2a-a9600932c161 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:48.223310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:48.224483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:55:48.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:48.233970 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 438/852] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:48.235116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6884-1715-4d93-9f2a-a9600932c161 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.263861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a51a73-a3b7-49fb-b145-f4da2fb1293b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:48.303396 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7642939-aff2-4ef2-9ee6-20c5c8e4fbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:48.304223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7642939-aff2-4ef2-9ee6-20c5c8e4fbab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:48.304598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7642939-aff2-4ef2-9ee6-20c5c8e4fbab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:48.305067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7642939-aff2-4ef2-9ee6-20c5c8e4fbab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:48.320830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7642939-aff2-4ef2-9ee6-20c5c8e4fbab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:48.322508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7642939-aff2-4ef2-9ee6-20c5c8e4fbab None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:48.333395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c7642939-aff2-4ef2-9ee6-20c5c8e4fbab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:48.343295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:48.344495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:48.345511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:48.345511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:48.345511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:48.352982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 415/853] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:48.354084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7642939-aff2-4ef2-9ee6-20c5c8e4fbab None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.372135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6884-1715-4d93-9f2a-a9600932c161 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:48.405523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:48.416900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4c4ff-5497-4a14-8043-d210e24f66a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:48.417833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4c4ff-5497-4a14-8043-d210e24f66a4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:48.418315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4c4ff-5497-4a14-8043-d210e24f66a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:48.419745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4c4ff-5497-4a14-8043-d210e24f66a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:48.420706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4c4ff-5497-4a14-8043-d210e24f66a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:48.421623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4c4ff-5497-4a14-8043-d210e24f66a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:48.423384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4c4ff-5497-4a14-8043-d210e24f66a4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:48.441253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 439/854] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:55:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:48.442619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4c4ff-5497-4a14-8043-d210e24f66a4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.482161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7642939-aff2-4ef2-9ee6-20c5c8e4fbab None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:48.526927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1116b-db40-4aa1-8433-a481e38dc279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:48.528117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1116b-db40-4aa1-8433-a481e38dc279 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:48.528899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1116b-db40-4aa1-8433-a481e38dc279 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:48.529389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1116b-db40-4aa1-8433-a481e38dc279 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:48.531087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1116b-db40-4aa1-8433-a481e38dc279 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:48.531606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1116b-db40-4aa1-8433-a481e38dc279 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:48.533191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1116b-db40-4aa1-8433-a481e38dc279 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:48.549451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 416/855] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:48.555321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1116b-db40-4aa1-8433-a481e38dc279 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.569963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4c4ff-5497-4a14-8043-d210e24f66a4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:48.613979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a5470-8ff7-42ed-93eb-d58d55b21acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:48.614628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a5470-8ff7-42ed-93eb-d58d55b21acb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:48.614907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a5470-8ff7-42ed-93eb-d58d55b21acb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:48.615179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a5470-8ff7-42ed-93eb-d58d55b21acb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:48.628914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a5470-8ff7-42ed-93eb-d58d55b21acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:48.630770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06a5470-8ff7-42ed-93eb-d58d55b21acb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:48.636099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f06a5470-8ff7-42ed-93eb-d58d55b21acb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:48.648845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:48.649870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:48.650943 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:48.650943 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:48.650943 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:48.654143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 440/856] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:48.656914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a5470-8ff7-42ed-93eb-d58d55b21acb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.694837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1116b-db40-4aa1-8433-a481e38dc279 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:48.714393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:48.744563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901101a-da90-44fb-939e-42db3375e8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:48.745209 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901101a-da90-44fb-939e-42db3375e8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:48.745515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901101a-da90-44fb-939e-42db3375e8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:48.745960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901101a-da90-44fb-939e-42db3375e8a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:48.756956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901101a-da90-44fb-939e-42db3375e8a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:48.759508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901101a-da90-44fb-939e-42db3375e8a7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3901101a-da90-44fb-939e-42db3375e8a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:48.776260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:48.777256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:48.778304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:48.778304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:48.778304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:48.778304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:48.778304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:48.778304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:48.778304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:48.778304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:48.778304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 19 20:55:48.778304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:48.781914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 417/857] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:55:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:48.785913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901101a-da90-44fb-939e-42db3375e8a7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.799822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a5470-8ff7-42ed-93eb-d58d55b21acb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:48.838818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee6511-565a-48da-9f52-0663a917a2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:48.839434 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee6511-565a-48da-9f52-0663a917a2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:48.839735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee6511-565a-48da-9f52-0663a917a2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:48.839984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee6511-565a-48da-9f52-0663a917a2e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:48.840621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee6511-565a-48da-9f52-0663a917a2e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:48.840921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee6511-565a-48da-9f52-0663a917a2e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:48.843574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee6511-565a-48da-9f52-0663a917a2e8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:48.861514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 441/858] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:48.862746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee6511-565a-48da-9f52-0663a917a2e8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.898329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901101a-da90-44fb-939e-42db3375e8a7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:48.920920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:48.936717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62afeb-0043-41a2-b53f-79dcfcbabf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:48.937354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62afeb-0043-41a2-b53f-79dcfcbabf7f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:48.937668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62afeb-0043-41a2-b53f-79dcfcbabf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:48.938024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62afeb-0043-41a2-b53f-79dcfcbabf7f None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:48.938744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62afeb-0043-41a2-b53f-79dcfcbabf7f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:48.939058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62afeb-0043-41a2-b53f-79dcfcbabf7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:48.940545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62afeb-0043-41a2-b53f-79dcfcbabf7f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:48.960494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 418/859] 10.222.0.16 () {56 vars in 1135 bytes} [Fri Aug 19 20:55:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:48.961791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0699fe6-346a-490b-95ba-2b364c93cbc9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:48.962084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0699fe6-346a-490b-95ba-2b364c93cbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:48.962362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0699fe6-346a-490b-95ba-2b364c93cbc9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:48.966023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 419/860] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:48.966686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0699fe6-346a-490b-95ba-2b364c93cbc9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:48.986829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:48.990413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee6511-565a-48da-9f52-0663a917a2e8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:49.031065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f90a9-e801-4f07-9107-0b0a26e37d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:49.032214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f90a9-e801-4f07-9107-0b0a26e37d9c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:49.032699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f90a9-e801-4f07-9107-0b0a26e37d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:49.033531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f90a9-e801-4f07-9107-0b0a26e37d9c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:49.045534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f90a9-e801-4f07-9107-0b0a26e37d9c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:49.048420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7f90a9-e801-4f07-9107-0b0a26e37d9c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-eb7f90a9-e801-4f07-9107-0b0a26e37d9c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:49.065464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:49.066752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:49.067912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:49.067912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:49.067912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:49.067912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:49.067912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:49.067912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:49.067912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:49.067912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:49.067912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:55:49.067912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:49.075247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 442/861] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Aug 19 20:55:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:49.077840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43859f86-d3ca-488e-9222-e46519cb0826 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:49.078418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43859f86-d3ca-488e-9222-e46519cb0826 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:49.078841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43859f86-d3ca-488e-9222-e46519cb0826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:49.082506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0699fe6-346a-490b-95ba-2b364c93cbc9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:49.099737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-43859f86-d3ca-488e-9222-e46519cb0826 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:49.101092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-43859f86-d3ca-488e-9222-e46519cb0826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:49.123938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1a4d3-88d9-455e-866e-2e94b23777df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:49.124892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1a4d3-88d9-455e-866e-2e94b23777df None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:49.125454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1a4d3-88d9-455e-866e-2e94b23777df None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:49.125951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1a4d3-88d9-455e-866e-2e94b23777df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:49.175664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:49.179398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 443/862] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:49 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:49.180253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43859f86-d3ca-488e-9222-e46519cb0826 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:49.213369 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1a4d3-88d9-455e-866e-2e94b23777df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:49.215325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1a4d3-88d9-455e-866e-2e94b23777df None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:49.308127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:49.329246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43859f86-d3ca-488e-9222-e46519cb0826 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:49.339461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 420/863] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:48 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:55:49.346089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1a4d3-88d9-455e-866e-2e94b23777df None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:49.375372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa82f95-6c62-42c1-b50a-efc07db9a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:49.376453 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa82f95-6c62-42c1-b50a-efc07db9a187 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:49.376956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa82f95-6c62-42c1-b50a-efc07db9a187 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:49.377428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa82f95-6c62-42c1-b50a-efc07db9a187 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:49.378480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa82f95-6c62-42c1-b50a-efc07db9a187 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:49.379005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa82f95-6c62-42c1-b50a-efc07db9a187 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:49.380658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa82f95-6c62-42c1-b50a-efc07db9a187 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:49.400794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 444/864] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Aug 19 20:55:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:49.405965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa82f95-6c62-42c1-b50a-efc07db9a187 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:49.454741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:49.480671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1a4d3-88d9-455e-866e-2e94b23777df None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:49.515509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa82f95-6c62-42c1-b50a-efc07db9a187 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:49.519549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb376fc5-ee32-4808-893b-143cf35fb70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:49.520203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb376fc5-ee32-4808-893b-143cf35fb70f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:49.520482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb376fc5-ee32-4808-893b-143cf35fb70f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:49.520755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb376fc5-ee32-4808-893b-143cf35fb70f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:49.533060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb376fc5-ee32-4808-893b-143cf35fb70f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:49.534548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb376fc5-ee32-4808-893b-143cf35fb70f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-eb376fc5-ee32-4808-893b-143cf35fb70f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:49.549629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:49.550725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:49.551596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:49.551596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:49.551596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:49.551596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 421/865] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:49.551596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb376fc5-ee32-4808-893b-143cf35fb70f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:49.560319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaf537-69ec-44ed-b672-c5380e28c8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:49.560985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaf537-69ec-44ed-b672-c5380e28c8fd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:49.561262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaf537-69ec-44ed-b672-c5380e28c8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:49.561542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcaf537-69ec-44ed-b672-c5380e28c8fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:49.581965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:49.641466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaf537-69ec-44ed-b672-c5380e28c8fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:49.642979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcaf537-69ec-44ed-b672-c5380e28c8fd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:49.671876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb376fc5-ee32-4808-893b-143cf35fb70f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:49.702226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:49.710000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8540c-19d3-4f33-8360-18a21205d18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:49.710547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8540c-19d3-4f33-8360-18a21205d18e None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:49.710824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8540c-19d3-4f33-8360-18a21205d18e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:49.711098 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8540c-19d3-4f33-8360-18a21205d18e None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:49.711753 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8540c-19d3-4f33-8360-18a21205d18e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:49.713132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8540c-19d3-4f33-8360-18a21205d18e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:49.733819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 422/866] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:55:49 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:49.733819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8540c-19d3-4f33-8360-18a21205d18e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:49.776962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 445/867] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:49 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 20:55:49.778227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaf537-69ec-44ed-b672-c5380e28c8fd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:49.850151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:49.889135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8540c-19d3-4f33-8360-18a21205d18e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:49.917563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcaf537-69ec-44ed-b672-c5380e28c8fd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:49.933940 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb20e41-56ab-4518-8d66-186f50e89441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:49.934575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb20e41-56ab-4518-8d66-186f50e89441 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:49.934903 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb20e41-56ab-4518-8d66-186f50e89441 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:49.935181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb20e41-56ab-4518-8d66-186f50e89441 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:49.935927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb20e41-56ab-4518-8d66-186f50e89441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:49.936144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb20e41-56ab-4518-8d66-186f50e89441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:49.937529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb20e41-56ab-4518-8d66-186f50e89441 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:49.952998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 423/868] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:49.954274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c8258-e40f-431c-8536-9954d4810828 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:49.954561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c8258-e40f-431c-8536-9954d4810828 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:49.954837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c8258-e40f-431c-8536-9954d4810828 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:49.956282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 424/869] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:49.960996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c8258-e40f-431c-8536-9954d4810828 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:49.972954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78a194-bd34-4e02-8ac7-966fbd7a5bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:49.974088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78a194-bd34-4e02-8ac7-966fbd7a5bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:49.974585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78a194-bd34-4e02-8ac7-966fbd7a5bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:49.975084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78a194-bd34-4e02-8ac7-966fbd7a5bd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:49.994557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78a194-bd34-4e02-8ac7-966fbd7a5bd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:49.998424 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78a194-bd34-4e02-8ac7-966fbd7a5bd4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:50.018475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4a78a194-bd34-4e02-8ac7-966fbd7a5bd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:50.023713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:50.025278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:50.027192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:50.027192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:50.027192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:50.030421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 446/870] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:50.031542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a5dd3-a059-4525-ac3b-bcff8ca82847 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:50.031854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a5dd3-a059-4525-ac3b-bcff8ca82847 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:50.032114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a5dd3-a059-4525-ac3b-bcff8ca82847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:50.053620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cf3a5dd3-a059-4525-ac3b-bcff8ca82847 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:50.054869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cf3a5dd3-a059-4525-ac3b-bcff8ca82847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:50.093297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c8258-e40f-431c-8536-9954d4810828 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:50.141796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 447/871] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:50.142004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af22424-58bd-4c01-81bc-d51c7001d7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:50.142004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af22424-58bd-4c01-81bc-d51c7001d7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:50.142216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a5dd3-a059-4525-ac3b-bcff8ca82847 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:50.142302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af22424-58bd-4c01-81bc-d51c7001d7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:50.142302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af22424-58bd-4c01-81bc-d51c7001d7e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:50.151015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af22424-58bd-4c01-81bc-d51c7001d7e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:50.152640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af22424-58bd-4c01-81bc-d51c7001d7e2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3af22424-58bd-4c01-81bc-d51c7001d7e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:50.167345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:50.168294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:50.169830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:50.169830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:50.169830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:50.169830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:50.169830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:50.169830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:50.169830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:50.169830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:50.170460 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 425/872] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:50.175734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af22424-58bd-4c01-81bc-d51c7001d7e2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:50.180411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:50.269874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a5dd3-a059-4525-ac3b-bcff8ca82847 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:50.318117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af22424-58bd-4c01-81bc-d51c7001d7e2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:50.320306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3c9d-3641-4716-84f0-d3c1ef8c050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:50.321061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3c9d-3641-4716-84f0-d3c1ef8c050e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:50.321205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3c9d-3641-4716-84f0-d3c1ef8c050e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:50.321528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3c9d-3641-4716-84f0-d3c1ef8c050e None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:50.322208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3c9d-3641-4716-84f0-d3c1ef8c050e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:50.322510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3c9d-3641-4716-84f0-d3c1ef8c050e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:50.326377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3c9d-3641-4716-84f0-d3c1ef8c050e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:50.331739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:50.347163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 448/873] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:50.352540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3c9d-3641-4716-84f0-d3c1ef8c050e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:50.371576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ec6fe-0c5c-4907-a252-e939976f6ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:50.373328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ec6fe-0c5c-4907-a252-e939976f6ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:50.374001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ec6fe-0c5c-4907-a252-e939976f6ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:50.374546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ec6fe-0c5c-4907-a252-e939976f6ae7 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:50.375481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ec6fe-0c5c-4907-a252-e939976f6ae7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:50.376022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ec6fe-0c5c-4907-a252-e939976f6ae7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:50.377881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ec6fe-0c5c-4907-a252-e939976f6ae7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:50.392693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 426/874] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:50.398381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ec6fe-0c5c-4907-a252-e939976f6ae7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:50.469332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:50.481503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3c9d-3641-4716-84f0-d3c1ef8c050e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:50.509243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ec6fe-0c5c-4907-a252-e939976f6ae7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:50.521068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb8ed7-1226-4150-9bef-f852cad21769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:50.521751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb8ed7-1226-4150-9bef-f852cad21769 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:50.522053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb8ed7-1226-4150-9bef-f852cad21769 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:50.522322 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdb8ed7-1226-4150-9bef-f852cad21769 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:50.530143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb8ed7-1226-4150-9bef-f852cad21769 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:50.531592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdb8ed7-1226-4150-9bef-f852cad21769 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:50.543357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f3b75-9418-4040-a173-0a41625cba8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-abdb8ed7-1226-4150-9bef-f852cad21769 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:50.543666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:50.545085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:50.546518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:50.546518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:50.546518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:50.546518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:50.546518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:50.546518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:50.546518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:50.546518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:50.547112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f3b75-9418-4040-a173-0a41625cba8e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:50.547112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f3b75-9418-4040-a173-0a41625cba8e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:50.547112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f3b75-9418-4040-a173-0a41625cba8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:50.547310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 449/875] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:50.547310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb8ed7-1226-4150-9bef-f852cad21769 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:50.550874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f3b75-9418-4040-a173-0a41625cba8e None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=61a0906b8b134c178b4dece2891f1596) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:50.551212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f3b75-9418-4040-a173-0a41625cba8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:50.552689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f3b75-9418-4040-a173-0a41625cba8e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:50.558831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 427/876] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:50 2022] GET /identity/v3/role_assignments?user.id=61a0906b8b134c178b4dece2891f1596&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=8b350845f331423cb0494e3449b58310 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:50.559630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f3b75-9418-4040-a173-0a41625cba8e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:50.613642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:50.694356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdb8ed7-1226-4150-9bef-f852cad21769 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:50.718851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f3b75-9418-4040-a173-0a41625cba8e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:50.745013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a5e6a-1fa3-4933-bde4-d7f866570ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:50.745662 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a5e6a-1fa3-4933-bde4-d7f866570ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:50.745973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a5e6a-1fa3-4933-bde4-d7f866570ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:50.746247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a5e6a-1fa3-4933-bde4-d7f866570ab4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:50.746948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a5e6a-1fa3-4933-bde4-d7f866570ab4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:50.748482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a5e6a-1fa3-4933-bde4-d7f866570ab4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fb6a5e6a-1fa3-4933-bde4-d7f866570ab4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:50.758981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:50.760127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 20:55:50.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:50.761883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90e560-aed3-4ac7-94df-d605f296fcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:50.761972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 450/877] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:55:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:50.762041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90e560-aed3-4ac7-94df-d605f296fcb1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:50.762041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90e560-aed3-4ac7-94df-d605f296fcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:50.762041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90e560-aed3-4ac7-94df-d605f296fcb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:50.762041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90e560-aed3-4ac7-94df-d605f296fcb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:50.762316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90e560-aed3-4ac7-94df-d605f296fcb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:50.765379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90e560-aed3-4ac7-94df-d605f296fcb1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:50.772208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a5e6a-1fa3-4933-bde4-d7f866570ab4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:50.779582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 428/878] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:50.785338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90e560-aed3-4ac7-94df-d605f296fcb1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:50.793657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:50.906162 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:50.912147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a5e6a-1fa3-4933-bde4-d7f866570ab4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:50.919203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90e560-aed3-4ac7-94df-d605f296fcb1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:50.952383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e244c3-1bba-4895-b870-103fe0faed50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:50.953127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e244c3-1bba-4895-b870-103fe0faed50 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:50.953408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e244c3-1bba-4895-b870-103fe0faed50 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:50.953684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e244c3-1bba-4895-b870-103fe0faed50 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:50.954407 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e244c3-1bba-4895-b870-103fe0faed50 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:50.954714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e244c3-1bba-4895-b870-103fe0faed50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:50.956322 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e244c3-1bba-4895-b870-103fe0faed50 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:50.962290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 451/879] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:55:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:50.964946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a4694-2c87-4ffe-9ea2-89355132d443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:50.965661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a4694-2c87-4ffe-9ea2-89355132d443 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:50.966626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a4694-2c87-4ffe-9ea2-89355132d443 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:50.967019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a4694-2c87-4ffe-9ea2-89355132d443 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:50.968151 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a4694-2c87-4ffe-9ea2-89355132d443 None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=bdce5268fa7c466280dddc19869dce87) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:50.968534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a4694-2c87-4ffe-9ea2-89355132d443 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:50.969038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e244c3-1bba-4895-b870-103fe0faed50 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:50.970192 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a4694-2c87-4ffe-9ea2-89355132d443 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:50.979314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 429/880] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:50 2022] GET /identity/v3/role_assignments?user.id=bdce5268fa7c466280dddc19869dce87&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:51.072441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:51.105351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e244c3-1bba-4895-b870-103fe0faed50 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:51.148891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2832253-3621-4a65-b512-92f69c8bf5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:51.149513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2832253-3621-4a65-b512-92f69c8bf5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:51.149900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2832253-3621-4a65-b512-92f69c8bf5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:51.150189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2832253-3621-4a65-b512-92f69c8bf5b7 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:51.150842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2832253-3621-4a65-b512-92f69c8bf5b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:51.151146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2832253-3621-4a65-b512-92f69c8bf5b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:51.152529 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2832253-3621-4a65-b512-92f69c8bf5b7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:51.159050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 452/881] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:55:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:51.166874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a4694-2c87-4ffe-9ea2-89355132d443 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:51.296586 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a4694-2c87-4ffe-9ea2-89355132d443 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:51.333298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f086459-d446-41eb-9612-40873cad5ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:51.333881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f086459-d446-41eb-9612-40873cad5ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:51.334166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f086459-d446-41eb-9612-40873cad5ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:51.334446 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f086459-d446-41eb-9612-40873cad5ed0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:51.335174 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f086459-d446-41eb-9612-40873cad5ed0 None admin] RBAC: Adding query filter params (interface=public, service_id=8b44a8f7e0bd43b9b0038ea0ef63ebf4, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:51.335489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f086459-d446-41eb-9612-40873cad5ed0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:51.337057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f086459-d446-41eb-9612-40873cad5ed0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:51.346828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 430/882] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Aug 19 20:55:51 2022] GET /identity/v3/endpoints?service_id=8b44a8f7e0bd43b9b0038ea0ef63ebf4&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:51.352680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2832253-3621-4a65-b512-92f69c8bf5b7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:51.467859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2832253-3621-4a65-b512-92f69c8bf5b7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:51.503994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa8373-e6be-4293-86d3-243d2b0e3b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:51.504958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8373-e6be-4293-86d3-243d2b0e3b1c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:51.505463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8373-e6be-4293-86d3-243d2b0e3b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:51.505982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8373-e6be-4293-86d3-243d2b0e3b1c None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:51.506783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa8373-e6be-4293-86d3-243d2b0e3b1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:51.508368 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa8373-e6be-4293-86d3-243d2b0e3b1c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:51.515719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 453/883] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Aug 19 20:55:51 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 20:55:51.793336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec3c51-15ec-4cc8-961c-65532155fcd1 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:51.793874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec3c51-15ec-4cc8-961c-65532155fcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:51.793952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec3c51-15ec-4cc8-961c-65532155fcd1 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:51.796209 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 431/884] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:51.802260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8474efe-f5bd-4fc7-b5e5-e9bdf9a4be7b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:51.802550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8474efe-f5bd-4fc7-b5e5-e9bdf9a4be7b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:51.802820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8474efe-f5bd-4fc7-b5e5-e9bdf9a4be7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:51.817844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:51.824180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d8474efe-f5bd-4fc7-b5e5-e9bdf9a4be7b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:51.825534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d8474efe-f5bd-4fc7-b5e5-e9bdf9a4be7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:51.898203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 454/885] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:52.164888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec3c51-15ec-4cc8-961c-65532155fcd1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:52.186815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:52.276289 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec3c51-15ec-4cc8-961c-65532155fcd1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:52.317783 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33175fb6-585e-40e7-afca-86ec6a8e2001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:52.318484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33175fb6-585e-40e7-afca-86ec6a8e2001 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:52.318767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33175fb6-585e-40e7-afca-86ec6a8e2001 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:52.319079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33175fb6-585e-40e7-afca-86ec6a8e2001 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:52.319639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33175fb6-585e-40e7-afca-86ec6a8e2001 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:52.321009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33175fb6-585e-40e7-afca-86ec6a8e2001 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-33175fb6-585e-40e7-afca-86ec6a8e2001 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:52.330233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:52.331407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:52.332477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 432/886] 10.222.0.16 () {56 vars in 1132 bytes} [Fri Aug 19 20:55:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:52.337548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8474efe-f5bd-4fc7-b5e5-e9bdf9a4be7b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:52.442864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8474efe-f5bd-4fc7-b5e5-e9bdf9a4be7b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:52.478597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c036f00-b6e1-4ebd-ba00-a2ba71a4a0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:52.479210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c036f00-b6e1-4ebd-ba00-a2ba71a4a0cf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:52.479490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c036f00-b6e1-4ebd-ba00-a2ba71a4a0cf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:52.479767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c036f00-b6e1-4ebd-ba00-a2ba71a4a0cf None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:52.480401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c036f00-b6e1-4ebd-ba00-a2ba71a4a0cf None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:52.480706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c036f00-b6e1-4ebd-ba00-a2ba71a4a0cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:52.482523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c036f00-b6e1-4ebd-ba00-a2ba71a4a0cf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:52.484413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:52.488248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 455/887] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:55:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:52.493792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33175fb6-585e-40e7-afca-86ec6a8e2001 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:52.581361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:52.600796 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33175fb6-585e-40e7-afca-86ec6a8e2001 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:52.637009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebeaf95-61f6-4dab-818f-ee4bb5fa8a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:52.637625 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeaf95-61f6-4dab-818f-ee4bb5fa8a0f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:52.637985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeaf95-61f6-4dab-818f-ee4bb5fa8a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:52.638261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeaf95-61f6-4dab-818f-ee4bb5fa8a0f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:52.639012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebeaf95-61f6-4dab-818f-ee4bb5fa8a0f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:52.639322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebeaf95-61f6-4dab-818f-ee4bb5fa8a0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:52.640708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebeaf95-61f6-4dab-818f-ee4bb5fa8a0f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:52.646421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 433/888] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:55:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:52.652406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c036f00-b6e1-4ebd-ba00-a2ba71a4a0cf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:52.705790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d75300-f585-40ab-9907-8a342348c34d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:52.706400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d75300-f585-40ab-9907-8a342348c34d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:52.706933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d75300-f585-40ab-9907-8a342348c34d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:52.709583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 434/889] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:52.719018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1006f1-c3a2-4ed7-9881-768c8bb9a0b4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:52.719544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1006f1-c3a2-4ed7-9881-768c8bb9a0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:52.720088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1006f1-c3a2-4ed7-9881-768c8bb9a0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:52.746697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dc1006f1-c3a2-4ed7-9881-768c8bb9a0b4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:52.748037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dc1006f1-c3a2-4ed7-9881-768c8bb9a0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:52.815237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c036f00-b6e1-4ebd-ba00-a2ba71a4a0cf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:52.837945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 435/890] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:52.867815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8241f0-cd15-4748-9abf-d5707cff0df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:52.868365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8241f0-cd15-4748-9abf-d5707cff0df0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:52.868644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8241f0-cd15-4748-9abf-d5707cff0df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:52.869171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8241f0-cd15-4748-9abf-d5707cff0df0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:52.870685 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8241f0-cd15-4748-9abf-d5707cff0df0 None admin] RBAC: Adding query filter params (interface=public, service_id=45116634fb324989be73e3cbc8ab3ccb, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:52.870685 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8241f0-cd15-4748-9abf-d5707cff0df0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:52.871652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8241f0-cd15-4748-9abf-d5707cff0df0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:52.878942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 456/891] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Aug 19 20:55:52 2022] GET /identity/v3/endpoints?service_id=45116634fb324989be73e3cbc8ab3ccb&interface=public®ion_id=RegionOne => generated 194 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:52.884604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1006f1-c3a2-4ed7-9881-768c8bb9a0b4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:52.996084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:53.031346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1006f1-c3a2-4ed7-9881-768c8bb9a0b4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:53.070968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061e843-5ce5-4531-b6be-19c4c3bc3422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:53.071585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061e843-5ce5-4531-b6be-19c4c3bc3422 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:53.071873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061e843-5ce5-4531-b6be-19c4c3bc3422 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:53.072150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061e843-5ce5-4531-b6be-19c4c3bc3422 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:53.072773 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061e843-5ce5-4531-b6be-19c4c3bc3422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:53.074195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061e843-5ce5-4531-b6be-19c4c3bc3422 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:53.080964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 436/892] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Aug 19 20:55:52 2022] GET /identity/v3/services => generated 1268 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 20:55:53.171743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8241f0-cd15-4748-9abf-d5707cff0df0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:53.253446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:53.297908 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8241f0-cd15-4748-9abf-d5707cff0df0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:53.302085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2ec3f-5617-402b-b3b0-c0203722536a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:53.302938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2ec3f-5617-402b-b3b0-c0203722536a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:53.303719 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2ec3f-5617-402b-b3b0-c0203722536a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:53.307173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 437/893] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:53.313975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc8876-772e-405b-acd3-e346f16cfbdd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:53.313975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc8876-772e-405b-acd3-e346f16cfbdd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:53.313975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc8876-772e-405b-acd3-e346f16cfbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:53.338397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-11fc8876-772e-405b-acd3-e346f16cfbdd None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:53.340859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-11fc8876-772e-405b-acd3-e346f16cfbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:53.352260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc12cb-8e72-4c30-8df5-9ac0d0cd3cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:53.352920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc12cb-8e72-4c30-8df5-9ac0d0cd3cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:53.353262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc12cb-8e72-4c30-8df5-9ac0d0cd3cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:53.353545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc12cb-8e72-4c30-8df5-9ac0d0cd3cc9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:53.354183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc12cb-8e72-4c30-8df5-9ac0d0cd3cc9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:53.355755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc12cb-8e72-4c30-8df5-9ac0d0cd3cc9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-45fc12cb-8e72-4c30-8df5-9ac0d0cd3cc9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:53.367950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:53.369266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:53.370829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 457/894] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:55:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:53.377271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc12cb-8e72-4c30-8df5-9ac0d0cd3cc9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:53.460150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 438/895] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:55:53.519281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc12cb-8e72-4c30-8df5-9ac0d0cd3cc9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:53.551047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:53.561726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0b2f9-f85b-4ec2-83a4-4f60729e5edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:53.562726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0b2f9-f85b-4ec2-83a4-4f60729e5edf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:53.563235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0b2f9-f85b-4ec2-83a4-4f60729e5edf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:53.563712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0b2f9-f85b-4ec2-83a4-4f60729e5edf None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:53.564557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0b2f9-f85b-4ec2-83a4-4f60729e5edf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:53.565065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0b2f9-f85b-4ec2-83a4-4f60729e5edf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:53.566716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0b2f9-f85b-4ec2-83a4-4f60729e5edf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:53.572649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 458/896] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Aug 19 20:55:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:53.579865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc8876-772e-405b-acd3-e346f16cfbdd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:53.668612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:53.670085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb7843-470b-46cf-9299-2f4f0ab0370a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:53.670435 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb7843-470b-46cf-9299-2f4f0ab0370a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:53.670658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb7843-470b-46cf-9299-2f4f0ab0370a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:53.672334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 459/897] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:55:53.678073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a9ae8-7bf7-4594-a44a-7bbc141e9b77 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:53.679428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a9ae8-7bf7-4594-a44a-7bbc141e9b77 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:53.679720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a9ae8-7bf7-4594-a44a-7bbc141e9b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:53.701160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3f6a9ae8-7bf7-4594-a44a-7bbc141e9b77 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:53.706139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3f6a9ae8-7bf7-4594-a44a-7bbc141e9b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:53.712935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc8876-772e-405b-acd3-e346f16cfbdd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:53.758927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b44fd6-8f82-4cbf-9ee8-2538813af8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:53.759740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b44fd6-8f82-4cbf-9ee8-2538813af8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:53.760120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b44fd6-8f82-4cbf-9ee8-2538813af8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:53.760397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b44fd6-8f82-4cbf-9ee8-2538813af8b6 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:53.761042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b44fd6-8f82-4cbf-9ee8-2538813af8b6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:53.761525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b44fd6-8f82-4cbf-9ee8-2538813af8b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:53.763854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b44fd6-8f82-4cbf-9ee8-2538813af8b6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:53.769281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 439/898] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Aug 19 20:55:53 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:53.770458 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b44fd6-8f82-4cbf-9ee8-2538813af8b6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:53.788374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 460/899] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:55:53.914697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b44fd6-8f82-4cbf-9ee8-2538813af8b6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:53.959016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffe03e-b580-4b0d-abb3-550be7a7bc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:53.959672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffe03e-b580-4b0d-abb3-550be7a7bc0a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:53.959953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffe03e-b580-4b0d-abb3-550be7a7bc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:53.960228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecffe03e-b580-4b0d-abb3-550be7a7bc0a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:53.963223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:53.974291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecffe03e-b580-4b0d-abb3-550be7a7bc0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ecffe03e-b580-4b0d-abb3-550be7a7bc0a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:53.979131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:53.980909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:53.983736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 440/900] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Aug 19 20:55:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:53.987363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a9ae8-7bf7-4594-a44a-7bbc141e9b77 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:54.087549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffe03e-b580-4b0d-abb3-550be7a7bc0a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:54.109596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a9ae8-7bf7-4594-a44a-7bbc141e9b77 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:54.142270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:54.160635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ec225-bdd7-446f-aed6-2282b8f473c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:54.162123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ec225-bdd7-446f-aed6-2282b8f473c9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:54.162554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ec225-bdd7-446f-aed6-2282b8f473c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:54.163244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ec225-bdd7-446f-aed6-2282b8f473c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:54.164070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ec225-bdd7-446f-aed6-2282b8f473c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:54.164519 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ec225-bdd7-446f-aed6-2282b8f473c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:54.167052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6ec225-bdd7-446f-aed6-2282b8f473c9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:54.179523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 461/901] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Aug 19 20:55:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:54.184400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ec225-bdd7-446f-aed6-2282b8f473c9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:54.209959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecffe03e-b580-4b0d-abb3-550be7a7bc0a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:54.257516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622d966-5050-44b9-a948-5189971ecf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:54.258568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d966-5050-44b9-a948-5189971ecf28 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:54.259031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d966-5050-44b9-a948-5189971ecf28 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:54.259503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622d966-5050-44b9-a948-5189971ecf28 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:54.260277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622d966-5050-44b9-a948-5189971ecf28 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:54.261894 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622d966-5050-44b9-a948-5189971ecf28 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f622d966-5050-44b9-a948-5189971ecf28 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:54.272293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:54.273352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 19 20:55:54.274237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:54.277150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 441/902] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:55:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:54.284067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622d966-5050-44b9-a948-5189971ecf28 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:54.317994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ec225-bdd7-446f-aed6-2282b8f473c9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:54.360195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:54.362751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f6b91-2079-480b-8c08-2c9b7baccf78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:54.363784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f6b91-2079-480b-8c08-2c9b7baccf78 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:54.364250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f6b91-2079-480b-8c08-2c9b7baccf78 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:54.364751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f6b91-2079-480b-8c08-2c9b7baccf78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:54.380437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f6b91-2079-480b-8c08-2c9b7baccf78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:54.382857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f6b91-2079-480b-8c08-2c9b7baccf78 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3d0f6b91-2079-480b-8c08-2c9b7baccf78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:54.407366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:54.408363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:54.409243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:54.409243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:54.409243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:54.409243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 462/903] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:54.414472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f6b91-2079-480b-8c08-2c9b7baccf78 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:54.416161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622d966-5050-44b9-a948-5189971ecf28 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:54.455924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a72760-1fdd-4dab-9e2b-f165218ed086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:54.456535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a72760-1fdd-4dab-9e2b-f165218ed086 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:54.456931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a72760-1fdd-4dab-9e2b-f165218ed086 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:54.457206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a72760-1fdd-4dab-9e2b-f165218ed086 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:54.457779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:54.459035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a72760-1fdd-4dab-9e2b-f165218ed086 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:54.459497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a72760-1fdd-4dab-9e2b-f165218ed086 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:54.461099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a72760-1fdd-4dab-9e2b-f165218ed086 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:54.472990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 442/904] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:55:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:54.474166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a72760-1fdd-4dab-9e2b-f165218ed086 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:54.538516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f6b91-2079-480b-8c08-2c9b7baccf78 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:54.583727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d05b3-ab07-42a7-a06b-c79a22a80e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:54.584758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d05b3-ab07-42a7-a06b-c79a22a80e39 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:54.585120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d05b3-ab07-42a7-a06b-c79a22a80e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:54.585482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d05b3-ab07-42a7-a06b-c79a22a80e39 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:54.586457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d05b3-ab07-42a7-a06b-c79a22a80e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:54.586833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d05b3-ab07-42a7-a06b-c79a22a80e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:54.588131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a72760-1fdd-4dab-9e2b-f165218ed086 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:54.588262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d05b3-ab07-42a7-a06b-c79a22a80e39 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:54.603942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 463/905] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:54.609931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d05b3-ab07-42a7-a06b-c79a22a80e39 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:54.613899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:54.629392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5073258-ed7b-4de6-8482-5be51b39a86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:54.630308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5073258-ed7b-4de6-8482-5be51b39a86f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:54.630760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5073258-ed7b-4de6-8482-5be51b39a86f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:54.631177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5073258-ed7b-4de6-8482-5be51b39a86f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:54.631984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5073258-ed7b-4de6-8482-5be51b39a86f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:54.632442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5073258-ed7b-4de6-8482-5be51b39a86f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:54.634044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5073258-ed7b-4de6-8482-5be51b39a86f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:54.644191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 443/906] 10.222.0.16 () {56 vars in 1114 bytes} [Fri Aug 19 20:55:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:54.645307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5073258-ed7b-4de6-8482-5be51b39a86f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:54.739735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:54.749090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d05b3-ab07-42a7-a06b-c79a22a80e39 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:54.773908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5073258-ed7b-4de6-8482-5be51b39a86f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:54.787518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1a50a-1745-440f-8e59-d9b76cbe7a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:54.788598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1a50a-1745-440f-8e59-d9b76cbe7a29 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:54.789081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1a50a-1745-440f-8e59-d9b76cbe7a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:54.789554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1a50a-1745-440f-8e59-d9b76cbe7a29 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:54.798272 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1a50a-1745-440f-8e59-d9b76cbe7a29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:54.800148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1a50a-1745-440f-8e59-d9b76cbe7a29 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-24b1a50a-1745-440f-8e59-d9b76cbe7a29 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:55:54.815177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:54.816154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:54.817124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:54.817124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:54.817124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:55:54.817124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:55:54.817124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:55:54.817124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:55:54.817124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:55:54.817124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:55:54.817124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:55:54.817124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:54.817674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881ba17-beee-4750-b98e-1ebb6f0ab8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:54.817674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881ba17-beee-4750-b98e-1ebb6f0ab8a5 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:54.817674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881ba17-beee-4750-b98e-1ebb6f0ab8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:54.817674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881ba17-beee-4750-b98e-1ebb6f0ab8a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:54.818672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881ba17-beee-4750-b98e-1ebb6f0ab8a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:54.820002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 464/907] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Aug 19 20:55:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:54.820247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881ba17-beee-4750-b98e-1ebb6f0ab8a5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:54.832457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1a50a-1745-440f-8e59-d9b76cbe7a29 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:54.857823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 444/908] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 20:55:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:54.961670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1a50a-1745-440f-8e59-d9b76cbe7a29 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:55.020580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49182df8-c3ec-42ee-9a62-450820b6d752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:55.021200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49182df8-c3ec-42ee-9a62-450820b6d752 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:55.021476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49182df8-c3ec-42ee-9a62-450820b6d752 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:55.021745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49182df8-c3ec-42ee-9a62-450820b6d752 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:55.022512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49182df8-c3ec-42ee-9a62-450820b6d752 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:55.022814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49182df8-c3ec-42ee-9a62-450820b6d752 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:55.024244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49182df8-c3ec-42ee-9a62-450820b6d752 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:55.049223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 465/909] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Aug 19 20:55:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:55.055803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881ba17-beee-4750-b98e-1ebb6f0ab8a5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:55.114579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:55.178125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881ba17-beee-4750-b98e-1ebb6f0ab8a5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:55.215532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa62846-b676-48b4-abef-c58421f871d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:55.216579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa62846-b676-48b4-abef-c58421f871d2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:55.217061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa62846-b676-48b4-abef-c58421f871d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:55.217548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa62846-b676-48b4-abef-c58421f871d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:55.288724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26ca87-bbcd-4799-b589-2f0807f17453 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:55.289057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26ca87-bbcd-4799-b589-2f0807f17453 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:55.289320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26ca87-bbcd-4799-b589-2f0807f17453 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:55.290504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa62846-b676-48b4-abef-c58421f871d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:55.290701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 466/910] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:55:55.295738 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa62846-b676-48b4-abef-c58421f871d2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:55.296682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51874736-4d2a-4f5a-a1c0-73ee791fa36c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:55.297020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51874736-4d2a-4f5a-a1c0-73ee791fa36c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:55.297377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51874736-4d2a-4f5a-a1c0-73ee791fa36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:55.317197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-51874736-4d2a-4f5a-a1c0-73ee791fa36c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:55.318380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-51874736-4d2a-4f5a-a1c0-73ee791fa36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:55.321554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:55.342813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:55.403097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 445/911] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:55:55 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:55:55.409540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa62846-b676-48b4-abef-c58421f871d2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:55.410677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 467/912] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:55.517287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa62846-b676-48b4-abef-c58421f871d2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:55.555524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897586ed-18c8-4351-b2fd-9ef5f042cc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:55.556246 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897586ed-18c8-4351-b2fd-9ef5f042cc26 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:55.556520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897586ed-18c8-4351-b2fd-9ef5f042cc26 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:55.556791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897586ed-18c8-4351-b2fd-9ef5f042cc26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:55.568630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897586ed-18c8-4351-b2fd-9ef5f042cc26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:55.570110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897586ed-18c8-4351-b2fd-9ef5f042cc26 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:55.572050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-897586ed-18c8-4351-b2fd-9ef5f042cc26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:55.588255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:55.589232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:55.590774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:55.590774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:55.590774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:55.590774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 446/913] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:55.595788 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51874736-4d2a-4f5a-a1c0-73ee791fa36c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:55.694534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897586ed-18c8-4351-b2fd-9ef5f042cc26 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:55.711987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51874736-4d2a-4f5a-a1c0-73ee791fa36c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:55.752062 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fc39-578e-4874-9a38-a1029ddb0d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:55.752955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fc39-578e-4874-9a38-a1029ddb0d90 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:55.753383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fc39-578e-4874-9a38-a1029ddb0d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:55.753861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16fc39-578e-4874-9a38-a1029ddb0d90 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:55.754697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16fc39-578e-4874-9a38-a1029ddb0d90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:55.755177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16fc39-578e-4874-9a38-a1029ddb0d90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:55.756748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16fc39-578e-4874-9a38-a1029ddb0d90 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:55.767055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:55.776443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 468/914] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:55.778074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fc39-578e-4874-9a38-a1029ddb0d90 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:55.832023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897586ed-18c8-4351-b2fd-9ef5f042cc26 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:55.881569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8c83da-0f1d-4d4b-8854-e93a6318ad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:55.882987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c83da-0f1d-4d4b-8854-e93a6318ad08 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:55.883981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c83da-0f1d-4d4b-8854-e93a6318ad08 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:55.884730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8c83da-0f1d-4d4b-8854-e93a6318ad08 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:55.885828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c83da-0f1d-4d4b-8854-e93a6318ad08 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:55.888790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8c83da-0f1d-4d4b-8854-e93a6318ad08 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:55.893263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16fc39-578e-4874-9a38-a1029ddb0d90 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-af8c83da-0f1d-4d4b-8854-e93a6318ad08 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:55.901709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:55.902806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 19 20:55:55.904258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:55.905093 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 447/915] 10.222.0.16 () {56 vars in 1132 bytes} [Fri Aug 19 20:55:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:55.906426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd049f-5a72-4a50-a418-22beab339131 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:55.906708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd049f-5a72-4a50-a418-22beab339131 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:55.906983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd049f-5a72-4a50-a418-22beab339131 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:55.908613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 448/916] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:55.913699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd049f-5a72-4a50-a418-22beab339131 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:55.940851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3677a-f670-4a86-bd1c-4ee338820e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:55.941540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3677a-f670-4a86-bd1c-4ee338820e83 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:55.941811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3677a-f670-4a86-bd1c-4ee338820e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:55.942063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3677a-f670-4a86-bd1c-4ee338820e83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:55.949260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3677a-f670-4a86-bd1c-4ee338820e83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:55.950922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d3677a-f670-4a86-bd1c-4ee338820e83 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-17d3677a-f670-4a86-bd1c-4ee338820e83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:55.965161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:55.966451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:55.967539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:55.967539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:55.967539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:55.967539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:55.967539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:55.967539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:55.967539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:55.967539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:55.968723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 469/917] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:55.970009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb57a22-23e6-4820-b35f-615335589d30 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:55.970444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb57a22-23e6-4820-b35f-615335589d30 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:55.970891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb57a22-23e6-4820-b35f-615335589d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:55.991541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7cb57a22-23e6-4820-b35f-615335589d30 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:55.992419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:55.992745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7cb57a22-23e6-4820-b35f-615335589d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:56.053932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:56.055878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd049f-5a72-4a50-a418-22beab339131 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:56.069736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 470/918] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:56.071172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb57a22-23e6-4820-b35f-615335589d30 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:56.098868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27af8a-5668-44f0-ad19-eb68eb993d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:56.099623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27af8a-5668-44f0-ad19-eb68eb993d96 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:56.100005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27af8a-5668-44f0-ad19-eb68eb993d96 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:56.100382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27af8a-5668-44f0-ad19-eb68eb993d96 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:56.101125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27af8a-5668-44f0-ad19-eb68eb993d96 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:56.101530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27af8a-5668-44f0-ad19-eb68eb993d96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:56.103051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27af8a-5668-44f0-ad19-eb68eb993d96 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:56.109167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 449/919] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:55:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:55:56.114642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27af8a-5668-44f0-ad19-eb68eb993d96 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:56.214546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb57a22-23e6-4820-b35f-615335589d30 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:56.224714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27af8a-5668-44f0-ad19-eb68eb993d96 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:56.253480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:56.268543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e71be8-1db9-4f07-9cfd-f50856c737ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:56.269046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcc9b5-52a4-4cc2-a064-1055507f7155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:56.269427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71be8-1db9-4f07-9cfd-f50856c737ac None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:56.269833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71be8-1db9-4f07-9cfd-f50856c737ac None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:56.270184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcc9b5-52a4-4cc2-a064-1055507f7155 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:56.270340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e71be8-1db9-4f07-9cfd-f50856c737ac None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:56.270750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcc9b5-52a4-4cc2-a064-1055507f7155 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:56.271232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dcc9b5-52a4-4cc2-a064-1055507f7155 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:56.271466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e71be8-1db9-4f07-9cfd-f50856c737ac None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:56.271872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e71be8-1db9-4f07-9cfd-f50856c737ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:56.272402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcc9b5-52a4-4cc2-a064-1055507f7155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:56.272873 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcc9b5-52a4-4cc2-a064-1055507f7155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:56.273524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e71be8-1db9-4f07-9cfd-f50856c737ac None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:56.274613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dcc9b5-52a4-4cc2-a064-1055507f7155 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:56.285323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 450/920] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:55:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:56.286535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e71be8-1db9-4f07-9cfd-f50856c737ac None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:56.290122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 471/921] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:55:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:56.297339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcc9b5-52a4-4cc2-a064-1055507f7155 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:56.380103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:56.402971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e71be8-1db9-4f07-9cfd-f50856c737ac None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:56.436822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dcc9b5-52a4-4cc2-a064-1055507f7155 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:56.442112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929138e9-be38-4242-8971-06371e5b1ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:56.442729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929138e9-be38-4242-8971-06371e5b1ee9 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:56.443009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929138e9-be38-4242-8971-06371e5b1ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:56.443354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929138e9-be38-4242-8971-06371e5b1ee9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:56.443949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929138e9-be38-4242-8971-06371e5b1ee9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:56.445323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929138e9-be38-4242-8971-06371e5b1ee9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:56.471958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 451/922] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 20:55:56 2022] POST /identity/v3/endpoints => generated 347 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:56.473702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929138e9-be38-4242-8971-06371e5b1ee9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:56.491036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849b1bc-7611-4bdf-833f-e87b255ae20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:56.491665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849b1bc-7611-4bdf-833f-e87b255ae20f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:56.491937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849b1bc-7611-4bdf-833f-e87b255ae20f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:56.492220 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849b1bc-7611-4bdf-833f-e87b255ae20f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:56.493014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849b1bc-7611-4bdf-833f-e87b255ae20f None admin] RBAC: Adding query filter params (role.id=a850548ca8b0447bba66f8d63c407705, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=bdce5268fa7c466280dddc19869dce87) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:56.493328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849b1bc-7611-4bdf-833f-e87b255ae20f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:56.495346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849b1bc-7611-4bdf-833f-e87b255ae20f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:56.503471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 472/923] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:55:56 2022] GET /identity/v3/role_assignments?user.id=bdce5268fa7c466280dddc19869dce87&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a850548ca8b0447bba66f8d63c407705 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:56.540513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:56.624300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929138e9-be38-4242-8971-06371e5b1ee9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:56.669504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df95e9e3-ba73-4f97-9e3e-47649af1705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:56.671312 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95e9e3-ba73-4f97-9e3e-47649af1705e None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:56.671805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95e9e3-ba73-4f97-9e3e-47649af1705e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:56.672297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95e9e3-ba73-4f97-9e3e-47649af1705e None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:56.673169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95e9e3-ba73-4f97-9e3e-47649af1705e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:56.674902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df95e9e3-ba73-4f97-9e3e-47649af1705e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:56.693299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 452/924] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:55:56 2022] POST /identity/v3/services => generated 241 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:55:57.069604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddbcc0-2c81-4537-8213-ed95267d61e3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:57.070141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddbcc0-2c81-4537-8213-ed95267d61e3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:57.070248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddbcc0-2c81-4537-8213-ed95267d61e3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:57.071808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 473/925] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:57.077361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc1b39-730e-415d-b826-d1528cc792a9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:57.077887 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc1b39-730e-415d-b826-d1528cc792a9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:57.077982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc1b39-730e-415d-b826-d1528cc792a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:57.102210 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-23bc1b39-730e-415d-b826-d1528cc792a9 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:57.103421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-23bc1b39-730e-415d-b826-d1528cc792a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:57.180161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 453/926] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:57.456139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddbcc0-2c81-4537-8213-ed95267d61e3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:57.566802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddbcc0-2c81-4537-8213-ed95267d61e3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:57.578568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:57.611634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:57.612542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:57.613030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:57.613457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:57.614311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:57.615941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:57.618784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WARNING keystone.server.flask.common [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 19 20:55:57.638611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.sql.core [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 19 20:55:57.638611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 19 20:55:57.638611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [parameters: {'id': '0ce623c1a8d1464583c171c9ff45864e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 19 20:55:57.638611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81571) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ce623c1a8d1464583c171c9ff45864e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ce623c1a8d1464583c171c9ff45864e. Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 20:55:57.657436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 19 20:55:57.658503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application yield resource Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.rollback() Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.force_reraise() Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise self.value Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application session.commit() Aug 19 20:55:57.659686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._prepare_impl() Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.flush() Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._flush(objects) Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application compat.raise_( Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application flush_context.execute() Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 19 20:55:57.660643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rec.execute(self) Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 19 20:55:57.661645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 20:55:57.662555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [parameters: {'id': '0ce623c1a8d1464583c171c9ff45864e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:57.663591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:57.664537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:57.665595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:57.665595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 19 20:55:57.665595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 19 20:55:57.665595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:57.665595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:57.665595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 19 20:55:57.665595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 19 20:55:57.665595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ce623c1a8d1464583c171c9ff45864e. Aug 19 20:55:57.665595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:57.668118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 474/927] 10.222.0.16 () {60 vars in 1140 bytes} [Fri Aug 19 20:55:57 2022] POST /identity/v3/projects => generated 262 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:57.674736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc1b39-730e-415d-b826-d1528cc792a9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:57.705974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:57.788272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc1b39-730e-415d-b826-d1528cc792a9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:57.825425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8085c174-908c-4eb0-9e01-0d3a16ac6844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:57.826204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085c174-908c-4eb0-9e01-0d3a16ac6844 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:57.826483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085c174-908c-4eb0-9e01-0d3a16ac6844 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:57.826758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8085c174-908c-4eb0-9e01-0d3a16ac6844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:57.833933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085c174-908c-4eb0-9e01-0d3a16ac6844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:57.835419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8085c174-908c-4eb0-9e01-0d3a16ac6844 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8085c174-908c-4eb0-9e01-0d3a16ac6844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:55:57.847807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:57.848936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:57.850089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:55:57.850089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:55:57.850089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:55:57.850089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:55:57.850089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:55:57.850089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:55:57.850089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:55:57.850089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:57.850089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 454/928] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:57.855810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:57.974482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b9c04-884b-4416-acb4-05c6f3a8b1d0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:58.010145 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59842c6-dd76-4d95-974a-7c8250f23c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:58.011029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59842c6-dd76-4d95-974a-7c8250f23c6d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:58.011464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59842c6-dd76-4d95-974a-7c8250f23c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:58.011895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59842c6-dd76-4d95-974a-7c8250f23c6d None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:58.012799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59842c6-dd76-4d95-974a-7c8250f23c6d None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:58.013256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59842c6-dd76-4d95-974a-7c8250f23c6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:58.015052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59842c6-dd76-4d95-974a-7c8250f23c6d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:58.036948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 475/929] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:55:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:58.558230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df1cf-e7e9-4db4-8566-48527d6f2fbc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:58.558968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df1cf-e7e9-4db4-8566-48527d6f2fbc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:58.558968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7df1cf-e7e9-4db4-8566-48527d6f2fbc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:58.560501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 455/930] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:58.566349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68258f67-b09e-4928-bf51-067fc8192129 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:58.567046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68258f67-b09e-4928-bf51-067fc8192129 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:58.567046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68258f67-b09e-4928-bf51-067fc8192129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:58.587505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-68258f67-b09e-4928-bf51-067fc8192129 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:58.589067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-68258f67-b09e-4928-bf51-067fc8192129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:58.610151 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e3432-5d07-42d8-938d-f303d5fc3afa None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:58.610441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e3432-5d07-42d8-938d-f303d5fc3afa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:58.610691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e3432-5d07-42d8-938d-f303d5fc3afa None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:58.612337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 456/931] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:55:58.618019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851b678-4d86-4944-8983-e87a821234f2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:58.618327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851b678-4d86-4944-8983-e87a821234f2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:58.618597 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c851b678-4d86-4944-8983-e87a821234f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:58.628946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:58.644006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c851b678-4d86-4944-8983-e87a821234f2 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:58.645595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c851b678-4d86-4944-8983-e87a821234f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:58.714975 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 476/932] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:58.734602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 457/933] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:55:58.738098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11c85e-28ee-47af-943c-7164facde6ab None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:58.738391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11c85e-28ee-47af-943c-7164facde6ab None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:58.738673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11c85e-28ee-47af-943c-7164facde6ab None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:58.740313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 477/934] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:55:58.745906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6e05-970c-485a-ace2-d9d6e592afd9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:58.746196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6e05-970c-485a-ace2-d9d6e592afd9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:58.746479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ad6e05-970c-485a-ace2-d9d6e592afd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:58.766884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-69ad6e05-970c-485a-ace2-d9d6e592afd9 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:55:58.768163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-69ad6e05-970c-485a-ace2-d9d6e592afd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:55:58.786671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:58.847318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 458/935] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:55:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:55:59.008668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11c85e-28ee-47af-943c-7164facde6ab None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.027024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad6e05-970c-485a-ace2-d9d6e592afd9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.136997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11c85e-28ee-47af-943c-7164facde6ab None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:59.151961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ad6e05-970c-485a-ace2-d9d6e592afd9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:59.177757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502d0c9-a4e2-4c40-a63e-cb17bd23cbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:59.178414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502d0c9-a4e2-4c40-a63e-cb17bd23cbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:59.178693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502d0c9-a4e2-4c40-a63e-cb17bd23cbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:59.178966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502d0c9-a4e2-4c40-a63e-cb17bd23cbf8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:59.190682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502d0c9-a4e2-4c40-a63e-cb17bd23cbf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:55:59.190864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a6268-3802-4582-b107-39b61a8788a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:59.191487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a6268-3802-4582-b107-39b61a8788a6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:59.191761 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a6268-3802-4582-b107-39b61a8788a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:59.192030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a6268-3802-4582-b107-39b61a8788a6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:59.192615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a6268-3802-4582-b107-39b61a8788a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:59.194176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a6268-3802-4582-b107-39b61a8788a6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7502d0c9-a4e2-4c40-a63e-cb17bd23cbf8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:55:59.195492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:59.196397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:59.197261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 478/936] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:59.198720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502d0c9-a4e2-4c40-a63e-cb17bd23cbf8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-104a6268-3802-4582-b107-39b61a8788a6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:59.209619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:59.210856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:55:59.211703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 459/937] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:55:59.212483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a6268-3802-4582-b107-39b61a8788a6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.224591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:59.235292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:59.317653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502d0c9-a4e2-4c40-a63e-cb17bd23cbf8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:59.331738 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a6268-3802-4582-b107-39b61a8788a6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:59.355855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a526ef-65d2-4fe5-9291-283f1c2f38e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:59.356699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a526ef-65d2-4fe5-9291-283f1c2f38e3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:59.357046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a526ef-65d2-4fe5-9291-283f1c2f38e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:59.357332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a526ef-65d2-4fe5-9291-283f1c2f38e3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:59.357964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a526ef-65d2-4fe5-9291-283f1c2f38e3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:59.359613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a526ef-65d2-4fe5-9291-283f1c2f38e3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:59.366609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b14e65-503f-4020-9c3c-7cdceb5696d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:59.367215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b14e65-503f-4020-9c3c-7cdceb5696d1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:59.367493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b14e65-503f-4020-9c3c-7cdceb5696d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:59.367824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b14e65-503f-4020-9c3c-7cdceb5696d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:59.368420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b14e65-503f-4020-9c3c-7cdceb5696d1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-80a526ef-65d2-4fe5-9291-283f1c2f38e3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:59.368711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:59.369950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:55:59.374584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:59.375459 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b14e65-503f-4020-9c3c-7cdceb5696d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:59.375546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 479/938] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:59.375627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b14e65-503f-4020-9c3c-7cdceb5696d1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:59.375708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a526ef-65d2-4fe5-9291-283f1c2f38e3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.382976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 460/939] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:59.383871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b14e65-503f-4020-9c3c-7cdceb5696d1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.467748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:59.469826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:59.491231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a526ef-65d2-4fe5-9291-283f1c2f38e3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:59.497205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b14e65-503f-4020-9c3c-7cdceb5696d1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:59.531001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c3f03-e030-4092-8f16-ca53d376907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:59.531608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c3f03-e030-4092-8f16-ca53d376907b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:59.531888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c3f03-e030-4092-8f16-ca53d376907b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:59.532157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c3f03-e030-4092-8f16-ca53d376907b None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:59.532819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c3f03-e030-4092-8f16-ca53d376907b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:59.533146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c3f03-e030-4092-8f16-ca53d376907b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:59.534382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b4721-773a-4e06-8673-3dfeadfb8bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:59.534500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9c3f03-e030-4092-8f16-ca53d376907b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:59.534996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b4721-773a-4e06-8673-3dfeadfb8bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:59.535277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b4721-773a-4e06-8673-3dfeadfb8bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:59.535561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b4721-773a-4e06-8673-3dfeadfb8bd4 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:59.536203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b4721-773a-4e06-8673-3dfeadfb8bd4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:59.536514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b4721-773a-4e06-8673-3dfeadfb8bd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:59.537902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b4721-773a-4e06-8673-3dfeadfb8bd4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:59.539795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 480/940] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:59.540754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c3f03-e030-4092-8f16-ca53d376907b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.552378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 461/941] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:59.552378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b4721-773a-4e06-8673-3dfeadfb8bd4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.657289 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9c3f03-e030-4092-8f16-ca53d376907b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:59.662534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b4721-773a-4e06-8673-3dfeadfb8bd4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:59.696043 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ab9bc-8fac-442a-a7cd-ca77b709852d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:59.696788 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ab9bc-8fac-442a-a7cd-ca77b709852d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:59.697166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ab9bc-8fac-442a-a7cd-ca77b709852d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:59.697450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ab9bc-8fac-442a-a7cd-ca77b709852d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:59.705287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74695070-12f4-428e-ab5f-eee9897a0e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:59.705932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695070-12f4-428e-ab5f-eee9897a0e97 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:59.706210 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695070-12f4-428e-ab5f-eee9897a0e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:59.706483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695070-12f4-428e-ab5f-eee9897a0e97 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:59.707120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74695070-12f4-428e-ab5f-eee9897a0e97 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:59.707418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74695070-12f4-428e-ab5f-eee9897a0e97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:59.708784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74695070-12f4-428e-ab5f-eee9897a0e97 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:59.710376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ab9bc-8fac-442a-a7cd-ca77b709852d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:59.712141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ab9bc-8fac-442a-a7cd-ca77b709852d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:59.713402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 462/942] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:59.714213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74695070-12f4-428e-ab5f-eee9897a0e97 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.716899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:59.725252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b12ab9bc-8fac-442a-a7cd-ca77b709852d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:55:59.740460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:55:59.741663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:55:59.742678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:55:59.742678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:55:59.742678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:55:59.742678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 481/943] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:55:59.749479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ab9bc-8fac-442a-a7cd-ca77b709852d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.850627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74695070-12f4-428e-ab5f-eee9897a0e97 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:59.894303 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b77cd5-102f-4793-abb4-091612541cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:59.894925 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77cd5-102f-4793-abb4-091612541cea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:59.895204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77cd5-102f-4793-abb4-091612541cea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:59.895476 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b77cd5-102f-4793-abb4-091612541cea None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:59.896115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b77cd5-102f-4793-abb4-091612541cea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:59.896418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b77cd5-102f-4793-abb4-091612541cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:59.897042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ab9bc-8fac-442a-a7cd-ca77b709852d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:55:59.899112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b77cd5-102f-4793-abb4-091612541cea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:59.904807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 463/944] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:59.911541 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b77cd5-102f-4793-abb4-091612541cea None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.938174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601c542-8911-4e85-babe-f7ad91e79e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:55:59.939351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601c542-8911-4e85-babe-f7ad91e79e52 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:55:59.939635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601c542-8911-4e85-babe-f7ad91e79e52 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:55:59.940192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601c542-8911-4e85-babe-f7ad91e79e52 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:55:59.940857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601c542-8911-4e85-babe-f7ad91e79e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:55:59.941162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601c542-8911-4e85-babe-f7ad91e79e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:55:59.943524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601c542-8911-4e85-babe-f7ad91e79e52 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:55:59.960155 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 482/945] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:55:59.965737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601c542-8911-4e85-babe-f7ad91e79e52 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:55:59.989582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:00.036080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b77cd5-102f-4793-abb4-091612541cea None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:00.051539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:00.080787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c5b18-0178-4ca0-8960-d5b0ec385026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:00.082024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c5b18-0178-4ca0-8960-d5b0ec385026 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:00.082746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c5b18-0178-4ca0-8960-d5b0ec385026 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:00.083476 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730c5b18-0178-4ca0-8960-d5b0ec385026 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:00.084621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c5b18-0178-4ca0-8960-d5b0ec385026 None admin] RBAC: Adding query filter params (interface=public, service_id=7a09b1f3ce0544b3945d1eedbcb467a4, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:00.085185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c5b18-0178-4ca0-8960-d5b0ec385026 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:00.086835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730c5b18-0178-4ca0-8960-d5b0ec385026 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:00.099294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 464/946] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/endpoints?service_id=7a09b1f3ce0544b3945d1eedbcb467a4&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:00.100672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c5b18-0178-4ca0-8960-d5b0ec385026 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:00.112673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601c542-8911-4e85-babe-f7ad91e79e52 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:00.160328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246df7b8-aeff-43f5-a2ce-fd338abb9a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:00.160969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246df7b8-aeff-43f5-a2ce-fd338abb9a3a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:00.161247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246df7b8-aeff-43f5-a2ce-fd338abb9a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:00.161520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246df7b8-aeff-43f5-a2ce-fd338abb9a3a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:00.168914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246df7b8-aeff-43f5-a2ce-fd338abb9a3a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:00.170933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246df7b8-aeff-43f5-a2ce-fd338abb9a3a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-246df7b8-aeff-43f5-a2ce-fd338abb9a3a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:56:00.182864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:00.183849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:00.185577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:00.185577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:00.185577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:56:00.185577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:56:00.185577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:56:00.185577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:56:00.185577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:56:00.185577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:56:00.185577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:56:00.185577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:00.189488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 483/947] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Aug 19 20:55:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:00.190869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b914b-11bf-4e8e-a8e4-cef379f002a9 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:00.191153 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b914b-11bf-4e8e-a8e4-cef379f002a9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:00.191433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b914b-11bf-4e8e-a8e4-cef379f002a9 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:00.195993 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 484/948] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:00.195993 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b914b-11bf-4e8e-a8e4-cef379f002a9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:00.226228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730c5b18-0178-4ca0-8960-d5b0ec385026 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:00.257651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:00.265636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e7091-78db-4b54-8710-0afb497d013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:00.266297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e7091-78db-4b54-8710-0afb497d013a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:00.266572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e7091-78db-4b54-8710-0afb497d013a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:00.266852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e7091-78db-4b54-8710-0afb497d013a None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:00.267465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e7091-78db-4b54-8710-0afb497d013a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:00.268864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83e7091-78db-4b54-8710-0afb497d013a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:00.281798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 465/949] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Aug 19 20:56:00 2022] GET /identity/v3/services => generated 1497 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 20:56:00.281798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152790b3-1016-483f-b0ec-a198a1a0c4ec None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:00.281798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152790b3-1016-483f-b0ec-a198a1a0c4ec None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:00.281798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152790b3-1016-483f-b0ec-a198a1a0c4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:00.298374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-152790b3-1016-483f-b0ec-a198a1a0c4ec None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:00.299619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-152790b3-1016-483f-b0ec-a198a1a0c4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:00.331048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b914b-11bf-4e8e-a8e4-cef379f002a9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:00.351118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:00.375559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844217d0-69c2-4faf-9cdc-1cc41f74e5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:00.376561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844217d0-69c2-4faf-9cdc-1cc41f74e5dc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:00.376561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844217d0-69c2-4faf-9cdc-1cc41f74e5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:00.378207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844217d0-69c2-4faf-9cdc-1cc41f74e5dc None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:00.379032 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844217d0-69c2-4faf-9cdc-1cc41f74e5dc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:00.379350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844217d0-69c2-4faf-9cdc-1cc41f74e5dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:00.381270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844217d0-69c2-4faf-9cdc-1cc41f74e5dc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:00.420392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 485/950] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Aug 19 20:56:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:00.423342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 466/951] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:00.424320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844217d0-69c2-4faf-9cdc-1cc41f74e5dc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:00.541338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844217d0-69c2-4faf-9cdc-1cc41f74e5dc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:00.578862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aa306-3e95-4dae-843b-1fcacfb99406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:00.579509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aa306-3e95-4dae-843b-1fcacfb99406 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:00.579784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aa306-3e95-4dae-843b-1fcacfb99406 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:00.580054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72aa306-3e95-4dae-843b-1fcacfb99406 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:00.591111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72aa306-3e95-4dae-843b-1fcacfb99406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:00.592635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72aa306-3e95-4dae-843b-1fcacfb99406 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d72aa306-3e95-4dae-843b-1fcacfb99406 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:00.608811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:00.610005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:56:00.611109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:00.611109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:00.611109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:00.611109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 486/952] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:00.617371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152790b3-1016-483f-b0ec-a198a1a0c4ec None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:00.712770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aa306-3e95-4dae-843b-1fcacfb99406 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:00.733042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152790b3-1016-483f-b0ec-a198a1a0c4ec None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:00.750152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:00.759226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:00.772352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955f271-a782-4f43-b3e1-666e97f77dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:00.772976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955f271-a782-4f43-b3e1-666e97f77dfa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:00.773256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955f271-a782-4f43-b3e1-666e97f77dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:00.773533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955f271-a782-4f43-b3e1-666e97f77dfa None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:00.774669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955f271-a782-4f43-b3e1-666e97f77dfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:00.775177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955f271-a782-4f43-b3e1-666e97f77dfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:00.777839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955f271-a782-4f43-b3e1-666e97f77dfa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:00.791700 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 467/953] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:00.798277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955f271-a782-4f43-b3e1-666e97f77dfa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:00.838784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72aa306-3e95-4dae-843b-1fcacfb99406 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:00.875833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a17876-0796-462f-813c-2503fffd9773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:00.876702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a17876-0796-462f-813c-2503fffd9773 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:00.876983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a17876-0796-462f-813c-2503fffd9773 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:00.877261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a17876-0796-462f-813c-2503fffd9773 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:00.877949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a17876-0796-462f-813c-2503fffd9773 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:00.879387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a17876-0796-462f-813c-2503fffd9773 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:00.898845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.sql.core [None req-16a17876-0796-462f-813c-2503fffd9773 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 19 20:56:00.898845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 19 20:56:00.898845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [parameters: {'user_id': 'ea0dfa97c81b4c409f85f06b2944444c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 19 20:56:00.898845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81571) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-16a17876-0796-462f-813c-2503fffd9773 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Aug 19 20:56:00.901477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 20:56:00.902391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application yield resource Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.rollback() Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.force_reraise() Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise self.value Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application session.commit() Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 19 20:56:00.903914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._prepare_impl() Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.flush() Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._flush(objects) Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application compat.raise_( Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application flush_context.execute() Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rec.execute(self) Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 19 20:56:00.905040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 19 20:56:00.906780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 19 20:56:00.907827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ea0dfa97c81b4c409f85f06b2944444c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:00.908755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 19 20:56:00.909844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 19 20:56:00.911532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 19 20:56:00.911532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 19 20:56:00.911532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 19 20:56:00.911532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 19 20:56:00.911532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 19 20:56:00.911532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:00.911532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 487/954] 10.222.0.16 () {60 vars in 1131 bytes} [Fri Aug 19 20:56:00 2022] POST /identity/v3/users => generated 160 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:00.911532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a17876-0796-462f-813c-2503fffd9773 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:00.923234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955f271-a782-4f43-b3e1-666e97f77dfa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:00.964529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331354c6-1120-4665-9aa1-0dd215ba11b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:00.965592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331354c6-1120-4665-9aa1-0dd215ba11b5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:00.966549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331354c6-1120-4665-9aa1-0dd215ba11b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:00.966953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331354c6-1120-4665-9aa1-0dd215ba11b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:00.975080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331354c6-1120-4665-9aa1-0dd215ba11b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:00.976759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331354c6-1120-4665-9aa1-0dd215ba11b5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-331354c6-1120-4665-9aa1-0dd215ba11b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:56:00.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:00.992712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:00.994451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:56:00.994451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:56:00.994451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:56:00.994451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:56:00.994451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:56:00.994451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:56:00.994451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:56:00.994451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:00.994451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 468/955] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:56:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:01.002662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331354c6-1120-4665-9aa1-0dd215ba11b5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:01.011176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:01.025706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a17876-0796-462f-813c-2503fffd9773 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:01.067951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2301ed-64ed-4513-99a2-7252282d76a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:01.068958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2301ed-64ed-4513-99a2-7252282d76a5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:01.069246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2301ed-64ed-4513-99a2-7252282d76a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:01.069883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2301ed-64ed-4513-99a2-7252282d76a5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:01.071441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:01.082626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2301ed-64ed-4513-99a2-7252282d76a5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:01.084876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2301ed-64ed-4513-99a2-7252282d76a5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1b2301ed-64ed-4513-99a2-7252282d76a5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:56:01.113799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:01.114986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:01.115921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:01.115921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:01.115921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:56:01.115921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:56:01.115921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:56:01.115921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:56:01.115921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:56:01.115921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:56:01.115921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 19 20:56:01.115921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:01.118884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 488/956] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:56:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:01.125746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2301ed-64ed-4513-99a2-7252282d76a5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:01.149420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331354c6-1120-4665-9aa1-0dd215ba11b5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:01.196416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d24a72-69e8-43f9-ae41-82548820d587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:01.197219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d24a72-69e8-43f9-ae41-82548820d587 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:01.197629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d24a72-69e8-43f9-ae41-82548820d587 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:01.198088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d24a72-69e8-43f9-ae41-82548820d587 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:01.199041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d24a72-69e8-43f9-ae41-82548820d587 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:01.199284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d24a72-69e8-43f9-ae41-82548820d587 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:01.202020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d24a72-69e8-43f9-ae41-82548820d587 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:01.218407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 469/957] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:56:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:01.226839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d24a72-69e8-43f9-ae41-82548820d587 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:01.268332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2301ed-64ed-4513-99a2-7252282d76a5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:01.322719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc53df1-439d-432d-8b12-9f3ab0fe288f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:01.324082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc53df1-439d-432d-8b12-9f3ab0fe288f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:01.324803 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc53df1-439d-432d-8b12-9f3ab0fe288f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:01.326465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc53df1-439d-432d-8b12-9f3ab0fe288f None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:01.328938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc53df1-439d-432d-8b12-9f3ab0fe288f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:01.330307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc53df1-439d-432d-8b12-9f3ab0fe288f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:01.334190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc53df1-439d-432d-8b12-9f3ab0fe288f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:01.361736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d24a72-69e8-43f9-ae41-82548820d587 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:01.364915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:01.377270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 489/958] 10.222.0.16 () {56 vars in 1101 bytes} [Fri Aug 19 20:56:01 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:01.409606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517af218-62a4-49df-93d4-c5494f10214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:01.410633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517af218-62a4-49df-93d4-c5494f10214d None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:01.411068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517af218-62a4-49df-93d4-c5494f10214d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:01.411493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517af218-62a4-49df-93d4-c5494f10214d None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/bdce5268fa7c466280dddc19869dce87/roles/a850548ca8b0447bba66f8d63c407705` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:01.414189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517af218-62a4-49df-93d4-c5494f10214d None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=bdce5268fa7c466280dddc19869dce87, role_id=a850548ca8b0447bba66f8d63c407705)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:01.415944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517af218-62a4-49df-93d4-c5494f10214d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:01.433025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 470/959] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:56:01 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/bdce5268fa7c466280dddc19869dce87/roles/a850548ca8b0447bba66f8d63c407705 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:56:01.817472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85395068-86ef-4898-bbe4-bfce0aed8be1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:01.818039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85395068-86ef-4898-bbe4-bfce0aed8be1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:01.818039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85395068-86ef-4898-bbe4-bfce0aed8be1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:01.819404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 490/960] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:01.824621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df5a43-a4f7-4c85-8ce2-c77bfe5b15ae None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:01.825666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df5a43-a4f7-4c85-8ce2-c77bfe5b15ae None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:01.825800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df5a43-a4f7-4c85-8ce2-c77bfe5b15ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:01.845112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-94df5a43-a4f7-4c85-8ce2-c77bfe5b15ae None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:01.846370 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-94df5a43-a4f7-4c85-8ce2-c77bfe5b15ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:01.925757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 471/961] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:02.199293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85395068-86ef-4898-bbe4-bfce0aed8be1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:02.209712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:02.315703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85395068-86ef-4898-bbe4-bfce0aed8be1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:02.353916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ecea7-8584-40e3-bdbe-6d9244f79686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:02.354419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ecea7-8584-40e3-bdbe-6d9244f79686 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:02.354697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ecea7-8584-40e3-bdbe-6d9244f79686 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:02.354960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ecea7-8584-40e3-bdbe-6d9244f79686 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:02.355628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ecea7-8584-40e3-bdbe-6d9244f79686 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:02.357745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ecea7-8584-40e3-bdbe-6d9244f79686 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:02.375684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 491/962] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:56:02 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:56:02.453509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9b5b1-d9c2-4aec-a23a-9ac2d782b83e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:02.454075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9b5b1-d9c2-4aec-a23a-9ac2d782b83e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:02.454433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b9b5b1-d9c2-4aec-a23a-9ac2d782b83e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:02.455974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 472/963] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:02.462196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824c36c-3cc6-4876-b44d-02b39b4ebf31 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:02.462548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824c36c-3cc6-4876-b44d-02b39b4ebf31 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:02.462748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2824c36c-3cc6-4876-b44d-02b39b4ebf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:02.483407 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2824c36c-3cc6-4876-b44d-02b39b4ebf31 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:02.484992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2824c36c-3cc6-4876-b44d-02b39b4ebf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:02.556833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:02.573403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 492/964] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:02.851284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9b5b1-d9c2-4aec-a23a-9ac2d782b83e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:02.953997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:02.964018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b9b5b1-d9c2-4aec-a23a-9ac2d782b83e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:02.999107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40796176-be91-4ee3-b667-92a34c850b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:02.999786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40796176-be91-4ee3-b667-92a34c850b1a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:03.000065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40796176-be91-4ee3-b667-92a34c850b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:03.000336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40796176-be91-4ee3-b667-92a34c850b1a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:03.000921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40796176-be91-4ee3-b667-92a34c850b1a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:03.002367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40796176-be91-4ee3-b667-92a34c850b1a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-40796176-be91-4ee3-b667-92a34c850b1a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:03.011329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:03.012343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:56:03.013848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:03.017227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 473/965] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:56:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:03.019502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824c36c-3cc6-4876-b44d-02b39b4ebf31 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:03.133068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2824c36c-3cc6-4876-b44d-02b39b4ebf31 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:03.169601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1c488-7690-43c9-8c55-571572acfde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:03.170750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1c488-7690-43c9-8c55-571572acfde9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:03.171329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1c488-7690-43c9-8c55-571572acfde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:03.171886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1c488-7690-43c9-8c55-571572acfde9 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:03.172738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1c488-7690-43c9-8c55-571572acfde9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:03.173256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1c488-7690-43c9-8c55-571572acfde9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:03.174854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1c488-7690-43c9-8c55-571572acfde9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:03.180342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 493/966] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Aug 19 20:56:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:03.186960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40796176-be91-4ee3-b667-92a34c850b1a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:03.291594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40796176-be91-4ee3-b667-92a34c850b1a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:03.328342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab7a7b-de28-4dd3-913d-dc866eb3002f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:03.329191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab7a7b-de28-4dd3-913d-dc866eb3002f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:03.329611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab7a7b-de28-4dd3-913d-dc866eb3002f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:03.330051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab7a7b-de28-4dd3-913d-dc866eb3002f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:03.330822 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab7a7b-de28-4dd3-913d-dc866eb3002f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:03.331330 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab7a7b-de28-4dd3-913d-dc866eb3002f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:03.332901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab7a7b-de28-4dd3-913d-dc866eb3002f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:03.338481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 474/967] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Aug 19 20:56:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:03.344333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1c488-7690-43c9-8c55-571572acfde9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:03.392063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaa61a-f0fb-4b99-94b4-1d77549e5e95 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:03.392256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaa61a-f0fb-4b99-94b4-1d77549e5e95 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:03.392535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaa61a-f0fb-4b99-94b4-1d77549e5e95 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:03.394028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 475/968] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:03.398748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c698e5e-4149-4b08-8a27-758fd4732ec4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:03.399033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c698e5e-4149-4b08-8a27-758fd4732ec4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:03.399309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c698e5e-4149-4b08-8a27-758fd4732ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:03.410012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:03.418143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c698e5e-4149-4b08-8a27-758fd4732ec4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:03.419901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c698e5e-4149-4b08-8a27-758fd4732ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:03.440029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:03.463511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1c488-7690-43c9-8c55-571572acfde9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:03.505807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 476/969] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:03.506039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683b985-2f6c-4d9c-b67f-b7d7b76eb57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:03.506039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683b985-2f6c-4d9c-b67f-b7d7b76eb57d None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:03.506039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683b985-2f6c-4d9c-b67f-b7d7b76eb57d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:03.506039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683b985-2f6c-4d9c-b67f-b7d7b76eb57d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:03.506325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683b985-2f6c-4d9c-b67f-b7d7b76eb57d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:03.506878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683b985-2f6c-4d9c-b67f-b7d7b76eb57d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:03.530120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 494/970] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 20:56:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:56:03.775190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c698e5e-4149-4b08-8a27-758fd4732ec4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:03.879456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c698e5e-4149-4b08-8a27-758fd4732ec4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:03.920529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82847a6-c705-4df6-8175-5e5e010ecb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:03.921174 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82847a6-c705-4df6-8175-5e5e010ecb61 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:03.921457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82847a6-c705-4df6-8175-5e5e010ecb61 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:03.921730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82847a6-c705-4df6-8175-5e5e010ecb61 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:03.932185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82847a6-c705-4df6-8175-5e5e010ecb61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-a82847a6-c705-4df6-8175-5e5e010ecb61 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:56:03.937572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:03.938548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:03.939381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 477/971] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Aug 19 20:56:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:03.946038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683b985-2f6c-4d9c-b67f-b7d7b76eb57d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:04.055871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683b985-2f6c-4d9c-b67f-b7d7b76eb57d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:04.070253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:04.097518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a02ff-a05b-4bd7-87a2-63086264bc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:04.098184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a02ff-a05b-4bd7-87a2-63086264bc93 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:04.098464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a02ff-a05b-4bd7-87a2-63086264bc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:04.098737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a02ff-a05b-4bd7-87a2-63086264bc93 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:04.099409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a02ff-a05b-4bd7-87a2-63086264bc93 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:04.099718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a02ff-a05b-4bd7-87a2-63086264bc93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:04.101081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a02ff-a05b-4bd7-87a2-63086264bc93 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:04.114666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 495/972] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Aug 19 20:56:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:04.115864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82847a6-c705-4df6-8175-5e5e010ecb61 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:04.128696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:04.228532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82847a6-c705-4df6-8175-5e5e010ecb61 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:04.264477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33887243-11ff-4576-b6dc-b3ff9eb3f4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:04.265111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33887243-11ff-4576-b6dc-b3ff9eb3f4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:04.265392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33887243-11ff-4576-b6dc-b3ff9eb3f4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:04.265664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33887243-11ff-4576-b6dc-b3ff9eb3f4f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:04.276041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33887243-11ff-4576-b6dc-b3ff9eb3f4f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:04.277491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33887243-11ff-4576-b6dc-b3ff9eb3f4f0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-33887243-11ff-4576-b6dc-b3ff9eb3f4f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:04.292286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:04.293253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:56:04.294821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:04.294821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:04.294821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:04.294821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 478/973] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:04.301131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a02ff-a05b-4bd7-87a2-63086264bc93 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:04.362116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae448-52e0-4e50-a71a-7a2c2b1c0562 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:04.362290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae448-52e0-4e50-a71a-7a2c2b1c0562 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:04.362583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae448-52e0-4e50-a71a-7a2c2b1c0562 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:04.364118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 479/974] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:04.368842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f3153-2e60-4b83-9e1a-68d6ccbd84bb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:04.369377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f3153-2e60-4b83-9e1a-68d6ccbd84bb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:04.369651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f3153-2e60-4b83-9e1a-68d6ccbd84bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:04.389432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b10f3153-2e60-4b83-9e1a-68d6ccbd84bb None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:04.391179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b10f3153-2e60-4b83-9e1a-68d6ccbd84bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:04.406308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a02ff-a05b-4bd7-87a2-63086264bc93 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:04.442399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaf485-cc4a-4927-8fa5-5b7fff2455c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:04.443008 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaf485-cc4a-4927-8fa5-5b7fff2455c1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:04.443287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaf485-cc4a-4927-8fa5-5b7fff2455c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:04.443568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceaf485-cc4a-4927-8fa5-5b7fff2455c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:04.444329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaf485-cc4a-4927-8fa5-5b7fff2455c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:04.444634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaf485-cc4a-4927-8fa5-5b7fff2455c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:04.446126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceaf485-cc4a-4927-8fa5-5b7fff2455c1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:04.451043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:04.464603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 496/975] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:04.465702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaf485-cc4a-4927-8fa5-5b7fff2455c1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:04.467344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 480/976] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:04 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:04.500280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:04.579388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceaf485-cc4a-4927-8fa5-5b7fff2455c1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:04.617871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f39d-2a78-4c7f-9478-71d60983b6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:04.618445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f39d-2a78-4c7f-9478-71d60983b6ca None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:04.618724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f39d-2a78-4c7f-9478-71d60983b6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:04.619251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61f39d-2a78-4c7f-9478-71d60983b6ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:04.628730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61f39d-2a78-4c7f-9478-71d60983b6ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:04.630266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61f39d-2a78-4c7f-9478-71d60983b6ca None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-be61f39d-2a78-4c7f-9478-71d60983b6ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:56:04.645026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:04.646374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:04.647867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 497/977] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Aug 19 20:56:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:04.651881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f3153-2e60-4b83-9e1a-68d6ccbd84bb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:04.740772 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f39d-2a78-4c7f-9478-71d60983b6ca None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:04.767230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f3153-2e60-4b83-9e1a-68d6ccbd84bb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:04.804253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcfb0d-a1ae-416d-bd32-c78718f860a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:04.805161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcfb0d-a1ae-416d-bd32-c78718f860a0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:04.805449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcfb0d-a1ae-416d-bd32-c78718f860a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:04.805796 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcfb0d-a1ae-416d-bd32-c78718f860a0 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:04.806606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcfb0d-a1ae-416d-bd32-c78718f860a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:04.806976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcfb0d-a1ae-416d-bd32-c78718f860a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:04.808425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcfb0d-a1ae-416d-bd32-c78718f860a0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:04.830151 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 481/978] 10.222.0.16 () {56 vars in 1149 bytes} [Fri Aug 19 20:56:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:04.835500 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcfb0d-a1ae-416d-bd32-c78718f860a0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:04.856423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:04.865667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61f39d-2a78-4c7f-9478-71d60983b6ca None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:04.891468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:04.907814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a6c7d-a5ef-44db-b281-cff2be6cec3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:04.908451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a6c7d-a5ef-44db-b281-cff2be6cec3d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:04.908733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a6c7d-a5ef-44db-b281-cff2be6cec3d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:04.909007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a6c7d-a5ef-44db-b281-cff2be6cec3d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:04.909600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a6c7d-a5ef-44db-b281-cff2be6cec3d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:04.911064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a6c7d-a5ef-44db-b281-cff2be6cec3d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dd5a6c7d-a5ef-44db-b281-cff2be6cec3d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:04.920636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:04.921731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:04.922599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 498/979] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:56:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:04.927654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a6c7d-a5ef-44db-b281-cff2be6cec3d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:04.957675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcfb0d-a1ae-416d-bd32-c78718f860a0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:04.994313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a3976-d218-4109-83b2-8b4bce0adf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:04.994930 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a3976-d218-4109-83b2-8b4bce0adf1a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:04.995208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a3976-d218-4109-83b2-8b4bce0adf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:04.995479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a3976-d218-4109-83b2-8b4bce0adf1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.050310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a6c7d-a5ef-44db-b281-cff2be6cec3d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:05.077872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a3976-d218-4109-83b2-8b4bce0adf1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:05.079518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04a3976-d218-4109-83b2-8b4bce0adf1a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:05.093887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874a468-970d-463e-ad69-14b91797e755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:05.094509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874a468-970d-463e-ad69-14b91797e755 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:05.094789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874a468-970d-463e-ad69-14b91797e755 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:05.095058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874a468-970d-463e-ad69-14b91797e755 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.095700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874a468-970d-463e-ad69-14b91797e755 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:05.096012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874a468-970d-463e-ad69-14b91797e755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:05.097400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874a468-970d-463e-ad69-14b91797e755 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:05.104215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 499/980] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:56:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:05.110064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874a468-970d-463e-ad69-14b91797e755 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:05.140507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:05.142967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:05.206592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 482/981] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:04 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 20:56:05.207765 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a3976-d218-4109-83b2-8b4bce0adf1a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:05.255962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874a468-970d-463e-ad69-14b91797e755 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:05.302072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55268df-b3c8-4415-999e-ae4393f456b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:05.302686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55268df-b3c8-4415-999e-ae4393f456b1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:05.302961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55268df-b3c8-4415-999e-ae4393f456b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:05.303237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55268df-b3c8-4415-999e-ae4393f456b1 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.303875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55268df-b3c8-4415-999e-ae4393f456b1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:05.304259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55268df-b3c8-4415-999e-ae4393f456b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:05.305525 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55268df-b3c8-4415-999e-ae4393f456b1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:05.312157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 500/982] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:56:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:05.317941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55268df-b3c8-4415-999e-ae4393f456b1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:05.340131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04a3976-d218-4109-83b2-8b4bce0adf1a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:05.384147 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deabb7-d857-4218-b5ea-be6cb093596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:05.384889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deabb7-d857-4218-b5ea-be6cb093596f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:05.385102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deabb7-d857-4218-b5ea-be6cb093596f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:05.385551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01deabb7-d857-4218-b5ea-be6cb093596f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.400572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01deabb7-d857-4218-b5ea-be6cb093596f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:05.402142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01deabb7-d857-4218-b5ea-be6cb093596f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:05.413118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:05.420430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-01deabb7-d857-4218-b5ea-be6cb093596f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:05.420658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:05.422200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:56:05.423212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:05.423212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:05.423212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:05.423212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 483/983] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:05.427750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deabb7-d857-4218-b5ea-be6cb093596f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:05.443706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55268df-b3c8-4415-999e-ae4393f456b1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:05.485911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf73410-b00a-4ada-a04f-24af6ec49d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:05.486552 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf73410-b00a-4ada-a04f-24af6ec49d15 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:05.486833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf73410-b00a-4ada-a04f-24af6ec49d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:05.487110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf73410-b00a-4ada-a04f-24af6ec49d15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.487834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf73410-b00a-4ada-a04f-24af6ec49d15 None admin] RBAC: Adding query filter params (interface=public, service_id=533e28b389684178b00362eb9c875576, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:05.488147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf73410-b00a-4ada-a04f-24af6ec49d15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:05.490205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf73410-b00a-4ada-a04f-24af6ec49d15 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:05.496349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 501/984] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Aug 19 20:56:05 2022] GET /identity/v3/endpoints?service_id=533e28b389684178b00362eb9c875576&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:05.502342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf73410-b00a-4ada-a04f-24af6ec49d15 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:05.548342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01deabb7-d857-4218-b5ea-be6cb093596f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:05.581616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153127e-1f7e-4ec2-a16a-520355ba4b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:05.582247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153127e-1f7e-4ec2-a16a-520355ba4b28 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:05.582530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153127e-1f7e-4ec2-a16a-520355ba4b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:05.582796 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153127e-1f7e-4ec2-a16a-520355ba4b28 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.583438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153127e-1f7e-4ec2-a16a-520355ba4b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:05.583737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153127e-1f7e-4ec2-a16a-520355ba4b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:05.585103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153127e-1f7e-4ec2-a16a-520355ba4b28 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:05.596811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 484/985] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:05.598063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c2bd7-3450-43b5-9768-2df34b2e8de4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:05.598345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c2bd7-3450-43b5-9768-2df34b2e8de4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:05.598622 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c2bd7-3450-43b5-9768-2df34b2e8de4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.599966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 485/986] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:05.602184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c2bd7-3450-43b5-9768-2df34b2e8de4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:05.611491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf73410-b00a-4ada-a04f-24af6ec49d15 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:05.645530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73910b4d-40fc-4906-96c8-721596dbda1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:05.646183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73910b4d-40fc-4906-96c8-721596dbda1d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:05.646457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73910b4d-40fc-4906-96c8-721596dbda1d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:05.646729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73910b4d-40fc-4906-96c8-721596dbda1d None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.647338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73910b4d-40fc-4906-96c8-721596dbda1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:05.648695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73910b4d-40fc-4906-96c8-721596dbda1d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:05.655032 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 502/987] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Aug 19 20:56:05 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 20:56:05.656159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d830c-6eb2-41f3-8584-69c19e0d3e6b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:05.656450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d830c-6eb2-41f3-8584-69c19e0d3e6b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:05.656723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d830c-6eb2-41f3-8584-69c19e0d3e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.668668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:05.676642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8b7d830c-6eb2-41f3-8584-69c19e0d3e6b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:05.677928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8b7d830c-6eb2-41f3-8584-69c19e0d3e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:05.689942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:05.723835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c2bd7-3450-43b5-9768-2df34b2e8de4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:05.763638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 503/988] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:05.766620 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422554b3-de0a-4f7c-adda-8e553d4d9d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:05.767360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422554b3-de0a-4f7c-adda-8e553d4d9d77 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:05.767644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422554b3-de0a-4f7c-adda-8e553d4d9d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:05.767917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422554b3-de0a-4f7c-adda-8e553d4d9d77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.777146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422554b3-de0a-4f7c-adda-8e553d4d9d77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:05.779917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422554b3-de0a-4f7c-adda-8e553d4d9d77 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-422554b3-de0a-4f7c-adda-8e553d4d9d77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:56:05.792814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:05.794707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:05.795880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:56:05.795880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:56:05.795880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:56:05.795880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:56:05.795880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:56:05.795880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:56:05.795880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:56:05.795880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:05.796820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 486/989] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:56:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:05.800775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d830c-6eb2-41f3-8584-69c19e0d3e6b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:05.907145 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d830c-6eb2-41f3-8584-69c19e0d3e6b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:05.943557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012833-4cb2-4666-abb2-d5a992c84dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:05.944153 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012833-4cb2-4666-abb2-d5a992c84dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:05.944430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012833-4cb2-4666-abb2-d5a992c84dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:05.944704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54012833-4cb2-4666-abb2-d5a992c84dc1 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:05.945403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012833-4cb2-4666-abb2-d5a992c84dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:05.945706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012833-4cb2-4666-abb2-d5a992c84dc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:05.947172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54012833-4cb2-4666-abb2-d5a992c84dc1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:05.959338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 504/990] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:56:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:05.966111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422554b3-de0a-4f7c-adda-8e553d4d9d77 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:06.034671 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012833-4cb2-4666-abb2-d5a992c84dc1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:06.038507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:06.082825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422554b3-de0a-4f7c-adda-8e553d4d9d77 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:06.104373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:06.119834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85b1f3-3b7d-4ff1-94f6-9a0399db6e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:06.120445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85b1f3-3b7d-4ff1-94f6-9a0399db6e3a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:06.120716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85b1f3-3b7d-4ff1-94f6-9a0399db6e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:06.120983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a85b1f3-3b7d-4ff1-94f6-9a0399db6e3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:06.121703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85b1f3-3b7d-4ff1-94f6-9a0399db6e3a None admin] RBAC: Adding query filter params (role.id=a850548ca8b0447bba66f8d63c407705, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=bdce5268fa7c466280dddc19869dce87) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:06.122088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85b1f3-3b7d-4ff1-94f6-9a0399db6e3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:06.123527 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a85b1f3-3b7d-4ff1-94f6-9a0399db6e3a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:06.137817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 487/991] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:56:05 2022] GET /identity/v3/role_assignments?user.id=bdce5268fa7c466280dddc19869dce87&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a850548ca8b0447bba66f8d63c407705 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:06.164339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54012833-4cb2-4666-abb2-d5a992c84dc1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:06.203373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2196bf-f877-4cdd-8ac4-5adb666202f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:06.204029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2196bf-f877-4cdd-8ac4-5adb666202f0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:06.204307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2196bf-f877-4cdd-8ac4-5adb666202f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:06.204577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2196bf-f877-4cdd-8ac4-5adb666202f0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:06.205172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2196bf-f877-4cdd-8ac4-5adb666202f0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:06.206620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2196bf-f877-4cdd-8ac4-5adb666202f0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3b2196bf-f877-4cdd-8ac4-5adb666202f0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:06.216328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:06.217518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:06.219240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 505/992] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:56:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:06.223649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85b1f3-3b7d-4ff1-94f6-9a0399db6e3a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:06.334132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a85b1f3-3b7d-4ff1-94f6-9a0399db6e3a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:06.369893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab621b4-4559-4d9a-aa18-60f105dc0bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:06.370505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab621b4-4559-4d9a-aa18-60f105dc0bfa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:06.370781 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab621b4-4559-4d9a-aa18-60f105dc0bfa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:06.371050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab621b4-4559-4d9a-aa18-60f105dc0bfa None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:06.371687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab621b4-4559-4d9a-aa18-60f105dc0bfa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:06.372085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab621b4-4559-4d9a-aa18-60f105dc0bfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:06.373592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab621b4-4559-4d9a-aa18-60f105dc0bfa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:06.378670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 488/993] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Aug 19 20:56:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:06.384602 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2196bf-f877-4cdd-8ac4-5adb666202f0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:06.456635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:06.494894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2196bf-f877-4cdd-8ac4-5adb666202f0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:06.530284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b73a4-41e9-4ab9-a1a8-2e06498f04b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:06.530899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b73a4-41e9-4ab9-a1a8-2e06498f04b7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:06.531178 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b73a4-41e9-4ab9-a1a8-2e06498f04b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:06.531451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b73a4-41e9-4ab9-a1a8-2e06498f04b7 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:06.532096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b73a4-41e9-4ab9-a1a8-2e06498f04b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:06.532400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b73a4-41e9-4ab9-a1a8-2e06498f04b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:06.533797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b73a4-41e9-4ab9-a1a8-2e06498f04b7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:06.543628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 506/994] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Aug 19 20:56:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:06.545347 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab621b4-4559-4d9a-aa18-60f105dc0bfa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:06.597168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:06.650432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab621b4-4559-4d9a-aa18-60f105dc0bfa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:06.687733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f52d41-fcd4-4902-bda2-891c5d073823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:06.688349 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f52d41-fcd4-4902-bda2-891c5d073823 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:06.688626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f52d41-fcd4-4902-bda2-891c5d073823 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:06.688897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f52d41-fcd4-4902-bda2-891c5d073823 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:06.689609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f52d41-fcd4-4902-bda2-891c5d073823 None admin] RBAC: Adding query filter params (interface=admin, service_id=7a09b1f3ce0544b3945d1eedbcb467a4, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:06.689954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f52d41-fcd4-4902-bda2-891c5d073823 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:06.691316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f52d41-fcd4-4902-bda2-891c5d073823 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:06.701485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 489/995] 10.222.0.16 () {56 vars in 1249 bytes} [Fri Aug 19 20:56:06 2022] GET /identity/v3/endpoints?service_id=7a09b1f3ce0544b3945d1eedbcb467a4&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:06.702518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b73a4-41e9-4ab9-a1a8-2e06498f04b7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:06.805468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b73a4-41e9-4ab9-a1a8-2e06498f04b7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:06.838696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:06.845746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c97ee-6f15-4837-9973-6e883bdc9202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:06.846545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c97ee-6f15-4837-9973-6e883bdc9202 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:06.846922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c97ee-6f15-4837-9973-6e883bdc9202 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:06.847294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c97ee-6f15-4837-9973-6e883bdc9202 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:06.847999 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c97ee-6f15-4837-9973-6e883bdc9202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:06.849467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c97ee-6f15-4837-9973-6e883bdc9202 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:06.856106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 507/996] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Aug 19 20:56:06 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 20:56:07.662593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da7f5f-94e0-4cd0-a01e-3dfac81bd4cf None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:07.663146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da7f5f-94e0-4cd0-a01e-3dfac81bd4cf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:07.663146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da7f5f-94e0-4cd0-a01e-3dfac81bd4cf None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:07.664609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 490/997] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:07.669817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4651855-eaf0-4a25-be0f-3da0416f14d6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:07.670223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4651855-eaf0-4a25-be0f-3da0416f14d6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:07.670397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4651855-eaf0-4a25-be0f-3da0416f14d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:07.688773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c4651855-eaf0-4a25-be0f-3da0416f14d6 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:07.689984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c4651855-eaf0-4a25-be0f-3da0416f14d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:07.760572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 508/998] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:08.030697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da7f5f-94e0-4cd0-a01e-3dfac81bd4cf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:08.096160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631692b1-737b-4a5a-b0ff-d5bc47a14cdf None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:08.096453 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631692b1-737b-4a5a-b0ff-d5bc47a14cdf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:08.096736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631692b1-737b-4a5a-b0ff-d5bc47a14cdf None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:08.098140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 509/999] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:56:08.103343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b421d-cbee-4603-bdb8-3ee11391a76d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:08.103629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b421d-cbee-4603-bdb8-3ee11391a76d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:08.103905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002b421d-cbee-4603-bdb8-3ee11391a76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:08.123724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-002b421d-cbee-4603-bdb8-3ee11391a76d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:08.124935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-002b421d-cbee-4603-bdb8-3ee11391a76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:08.168519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39da7f5f-94e0-4cd0-a01e-3dfac81bd4cf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:08.190575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:08.205392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 510/1000] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:08.219680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15fe35-e02e-4cf1-ab84-4f881bf7ba8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:08.220715 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15fe35-e02e-4cf1-ab84-4f881bf7ba8c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:08.221222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15fe35-e02e-4cf1-ab84-4f881bf7ba8c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:08.221726 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15fe35-e02e-4cf1-ab84-4f881bf7ba8c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:08.222886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15fe35-e02e-4cf1-ab84-4f881bf7ba8c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:08.225387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15fe35-e02e-4cf1-ab84-4f881bf7ba8c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2d15fe35-e02e-4cf1-ab84-4f881bf7ba8c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:08.239087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:08.240110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:08.242928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 491/1001] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:56:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:08.246425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b421d-cbee-4603-bdb8-3ee11391a76d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:08.317791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:08.367066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002b421d-cbee-4603-bdb8-3ee11391a76d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:08.406221 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc486-fcf8-4065-893d-ae048b8f059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:08.406838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc486-fcf8-4065-893d-ae048b8f059b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:08.407112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc486-fcf8-4065-893d-ae048b8f059b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:08.407382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cc486-fcf8-4065-893d-ae048b8f059b None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:08.408020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cc486-fcf8-4065-893d-ae048b8f059b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:08.408319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cc486-fcf8-4065-893d-ae048b8f059b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:08.409694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cc486-fcf8-4065-893d-ae048b8f059b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:08.421135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 511/1002] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:56:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:08.422222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15fe35-e02e-4cf1-ab84-4f881bf7ba8c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:08.481959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc486-fcf8-4065-893d-ae048b8f059b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:08.539363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15fe35-e02e-4cf1-ab84-4f881bf7ba8c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:08.580651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7bbf6-5004-43e5-90b2-6ccaaff045df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:08.581613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7bbf6-5004-43e5-90b2-6ccaaff045df None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:08.582136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7bbf6-5004-43e5-90b2-6ccaaff045df None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:08.582685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7bbf6-5004-43e5-90b2-6ccaaff045df None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:08.583634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7bbf6-5004-43e5-90b2-6ccaaff045df None admin] RBAC: Adding query filter params (type=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:08.584158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7bbf6-5004-43e5-90b2-6ccaaff045df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:08.585792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7bbf6-5004-43e5-90b2-6ccaaff045df None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:08.593136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 492/1003] 10.222.0.16 () {56 vars in 1112 bytes} [Fri Aug 19 20:56:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:08.598993 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7bbf6-5004-43e5-90b2-6ccaaff045df None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:08.605494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cc486-fcf8-4065-893d-ae048b8f059b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:08.631123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:08.647250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:08.648561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ea773-2112-4a55-ac02-18121a97c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:08.649206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ea773-2112-4a55-ac02-18121a97c346 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:08.649510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ea773-2112-4a55-ac02-18121a97c346 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:08.649898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ea773-2112-4a55-ac02-18121a97c346 None admin] PATH_INFO: `/v3/services/image` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:08.650495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ea773-2112-4a55-ac02-18121a97c346 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:08.651885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ea773-2112-4a55-ac02-18121a97c346 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-461ea773-2112-4a55-ac02-18121a97c346 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:08.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:08.665914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:08.666919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 512/1004] 10.222.0.16 () {56 vars in 1105 bytes} [Fri Aug 19 20:56:08 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:08.668086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ea773-2112-4a55-ac02-18121a97c346 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:08.725800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7bbf6-5004-43e5-90b2-6ccaaff045df None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:08.767249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ac97c3-d4de-4985-b85e-9c498d7c7c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:08.767860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac97c3-d4de-4985-b85e-9c498d7c7c92 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:08.768137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac97c3-d4de-4985-b85e-9c498d7c7c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:08.768412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac97c3-d4de-4985-b85e-9c498d7c7c92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:08.769077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac97c3-d4de-4985-b85e-9c498d7c7c92 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:08.770471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ac97c3-d4de-4985-b85e-9c498d7c7c92 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:08.788401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ea773-2112-4a55-ac02-18121a97c346 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:08.793685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 493/1005] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 20:56:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 20:56:08.818266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34d0c2-0445-49ea-ae38-24fe770f6a3e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:08.818602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34d0c2-0445-49ea-ae38-24fe770f6a3e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:08.818821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34d0c2-0445-49ea-ae38-24fe770f6a3e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:08.820217 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 494/1006] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:56:08.837386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250522-170d-481f-ba99-b51d59f0dacc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:08.837695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250522-170d-481f-ba99-b51d59f0dacc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:08.838230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250522-170d-481f-ba99-b51d59f0dacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:08.850975 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea10e93-af76-4329-95d3-b494318fe5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:08.851614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea10e93-af76-4329-95d3-b494318fe5eb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:08.851899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea10e93-af76-4329-95d3-b494318fe5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:08.852165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea10e93-af76-4329-95d3-b494318fe5eb None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:08.852845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea10e93-af76-4329-95d3-b494318fe5eb None admin] RBAC: Adding query filter params (name=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:08.853131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea10e93-af76-4329-95d3-b494318fe5eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:08.854580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea10e93-af76-4329-95d3-b494318fe5eb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:08.865470 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 513/1007] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:08.867515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea10e93-af76-4329-95d3-b494318fe5eb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:08.872468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6e250522-170d-481f-ba99-b51d59f0dacc None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:08.874915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6e250522-170d-481f-ba99-b51d59f0dacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:08.930047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:08.960551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 495/1008] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:08 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:08.998710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea10e93-af76-4329-95d3-b494318fe5eb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:09.042604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9957b-284b-4618-a75c-b36e099a2d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:09.043553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9957b-284b-4618-a75c-b36e099a2d5b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:09.044557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9957b-284b-4618-a75c-b36e099a2d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:09.044929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e9957b-284b-4618-a75c-b36e099a2d5b None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:09.045571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9957b-284b-4618-a75c-b36e099a2d5b None admin] RBAC: Adding query filter params (type=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:09.045904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9957b-284b-4618-a75c-b36e099a2d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:09.047276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e9957b-284b-4618-a75c-b36e099a2d5b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:09.053077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 514/1009] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:09.242081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e250522-170d-481f-ba99-b51d59f0dacc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:09.274878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:09.378284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e250522-170d-481f-ba99-b51d59f0dacc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:09.429518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d485e0f-8a02-48f6-bd10-c57c5c6b913e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:09.429518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d485e0f-8a02-48f6-bd10-c57c5c6b913e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:09.429518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d485e0f-8a02-48f6-bd10-c57c5c6b913e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:09.429518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d485e0f-8a02-48f6-bd10-c57c5c6b913e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:09.430410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d485e0f-8a02-48f6-bd10-c57c5c6b913e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:09.431038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d485e0f-8a02-48f6-bd10-c57c5c6b913e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4d485e0f-8a02-48f6-bd10-c57c5c6b913e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:09.443378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:09.444422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 19 20:56:09.445283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:09.449559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 496/1010] 10.222.0.16 () {56 vars in 1126 bytes} [Fri Aug 19 20:56:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:09.451568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9957b-284b-4618-a75c-b36e099a2d5b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:09.562034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e9957b-284b-4618-a75c-b36e099a2d5b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:09.572213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:09.600357 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fb915-868f-4c6e-becb-7320e2bbcbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:09.600972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fb915-868f-4c6e-becb-7320e2bbcbea None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:09.601253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fb915-868f-4c6e-becb-7320e2bbcbea None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:09.601526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093fb915-868f-4c6e-becb-7320e2bbcbea None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:09.602186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fb915-868f-4c6e-becb-7320e2bbcbea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:09.602492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fb915-868f-4c6e-becb-7320e2bbcbea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:09.603875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093fb915-868f-4c6e-becb-7320e2bbcbea None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:09.614227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 515/1011] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Aug 19 20:56:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:09.615850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d485e0f-8a02-48f6-bd10-c57c5c6b913e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:09.711399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:09.723077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d485e0f-8a02-48f6-bd10-c57c5c6b913e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:09.760539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fed4a-d996-40e8-8f2b-6d699f9b99a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:09.761180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fed4a-d996-40e8-8f2b-6d699f9b99a5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:09.761464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fed4a-d996-40e8-8f2b-6d699f9b99a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:09.761706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fed4a-d996-40e8-8f2b-6d699f9b99a5 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:09.762731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fed4a-d996-40e8-8f2b-6d699f9b99a5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:09.763052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fed4a-d996-40e8-8f2b-6d699f9b99a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:09.764684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fed4a-d996-40e8-8f2b-6d699f9b99a5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:09.774647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 497/1012] 10.222.0.16 () {56 vars in 1122 bytes} [Fri Aug 19 20:56:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:09.775646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fb915-868f-4c6e-becb-7320e2bbcbea None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:09.882583 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093fb915-868f-4c6e-becb-7320e2bbcbea None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:09.917158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc2570-c8f6-42ba-a911-53f759effb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:09.917803 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc2570-c8f6-42ba-a911-53f759effb50 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:09.918110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc2570-c8f6-42ba-a911-53f759effb50 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:09.918387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc2570-c8f6-42ba-a911-53f759effb50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:09.919957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc2570-c8f6-42ba-a911-53f759effb50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:09.921048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc2570-c8f6-42ba-a911-53f759effb50 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:09.939456 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 516/1013] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 20:56:09 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:56:10.920420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54d831-c5e0-4cd7-85e2-22f655785cc5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:10.921051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54d831-c5e0-4cd7-85e2-22f655785cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:10.921051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54d831-c5e0-4cd7-85e2-22f655785cc5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:10.922664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 498/1014] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:10.927914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa96608-5a22-4735-80df-6b58d8c9409d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:10.928415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa96608-5a22-4735-80df-6b58d8c9409d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:10.928521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa96608-5a22-4735-80df-6b58d8c9409d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:10.947946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5fa96608-5a22-4735-80df-6b58d8c9409d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:10.949146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5fa96608-5a22-4735-80df-6b58d8c9409d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:10.978142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:10.993577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb67cbc-a3ae-4746-b79a-fb1b180549c0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:10.993909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb67cbc-a3ae-4746-b79a-fb1b180549c0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:10.994435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb67cbc-a3ae-4746-b79a-fb1b180549c0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:10.995923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 499/1015] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:56:11.001059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f340933-8640-447c-be6a-7fc3cc82e008 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:11.001344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f340933-8640-447c-be6a-7fc3cc82e008 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:11.001637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f340933-8640-447c-be6a-7fc3cc82e008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:11.021130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0f340933-8640-447c-be6a-7fc3cc82e008 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:11.022367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0f340933-8640-447c-be6a-7fc3cc82e008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:11.034454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 517/1016] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:10 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:11.111606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 500/1017] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:10 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:11.314302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa96608-5a22-4735-80df-6b58d8c9409d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:11.380903 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f340933-8640-447c-be6a-7fc3cc82e008 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:11.423655 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa96608-5a22-4735-80df-6b58d8c9409d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:11.463188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac062bc1-798f-4677-89ae-f2a0509297de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:11.463825 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac062bc1-798f-4677-89ae-f2a0509297de None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:11.464096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac062bc1-798f-4677-89ae-f2a0509297de None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:11.464365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac062bc1-798f-4677-89ae-f2a0509297de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:11.475384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:11.476164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac062bc1-798f-4677-89ae-f2a0509297de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ac062bc1-798f-4677-89ae-f2a0509297de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:56:11.482630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:11.483747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:11.485501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 518/1018] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Aug 19 20:56:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:11.490521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac062bc1-798f-4677-89ae-f2a0509297de None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:11.498759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f340933-8640-447c-be6a-7fc3cc82e008 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:11.520988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:11.540189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00428-8dc3-45c5-82b4-89682256a76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:11.540436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00428-8dc3-45c5-82b4-89682256a76c None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:11.540649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00428-8dc3-45c5-82b4-89682256a76c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:11.540893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba00428-8dc3-45c5-82b4-89682256a76c None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:11.541563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba00428-8dc3-45c5-82b4-89682256a76c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:11.542977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba00428-8dc3-45c5-82b4-89682256a76c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:11.565896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 501/1019] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:56:11 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 20:56:11.623392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac062bc1-798f-4677-89ae-f2a0509297de None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:11.669809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe9a7f-ceba-4693-8a4e-2b49229ff651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:11.670428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe9a7f-ceba-4693-8a4e-2b49229ff651 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:11.670705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe9a7f-ceba-4693-8a4e-2b49229ff651 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:11.670978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe9a7f-ceba-4693-8a4e-2b49229ff651 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:11.671667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe9a7f-ceba-4693-8a4e-2b49229ff651 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:11.671970 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe9a7f-ceba-4693-8a4e-2b49229ff651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:11.673422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe9a7f-ceba-4693-8a4e-2b49229ff651 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:11.688292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 519/1020] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Aug 19 20:56:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:11.690328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00428-8dc3-45c5-82b4-89682256a76c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:11.810963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba00428-8dc3-45c5-82b4-89682256a76c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:11.848102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f80ad0f-d658-48ba-9fb0-ff6c91d14a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:11.848662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80ad0f-d658-48ba-9fb0-ff6c91d14a09 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:11.848935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80ad0f-d658-48ba-9fb0-ff6c91d14a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:11.849205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f80ad0f-d658-48ba-9fb0-ff6c91d14a09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:11.860713 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80ad0f-d658-48ba-9fb0-ff6c91d14a09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:11.862219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f80ad0f-d658-48ba-9fb0-ff6c91d14a09 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:11.864896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6f80ad0f-d658-48ba-9fb0-ff6c91d14a09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:11.880445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:11.881737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:56:11.883003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:11.883003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:11.883003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:11.884860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 502/1021] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:11.892189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe9a7f-ceba-4693-8a4e-2b49229ff651 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:11.915197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01940e05-84fd-4a08-8a7b-d6a0dfc80bcc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:11.915803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01940e05-84fd-4a08-8a7b-d6a0dfc80bcc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:11.916099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01940e05-84fd-4a08-8a7b-d6a0dfc80bcc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:11.917610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 503/1022] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:56:11.923159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d055fb-4fc5-4555-a11b-111d4165b793 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:11.923740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d055fb-4fc5-4555-a11b-111d4165b793 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:11.923740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d055fb-4fc5-4555-a11b-111d4165b793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:11.945069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-02d055fb-4fc5-4555-a11b-111d4165b793 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:11.946326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-02d055fb-4fc5-4555-a11b-111d4165b793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:11.992657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:12.025391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe9a7f-ceba-4693-8a4e-2b49229ff651 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:12.041607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 504/1023] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:11 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:12.069191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3991a4-9fe8-40eb-a050-4f013fc3dcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:12.069952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3991a4-9fe8-40eb-a050-4f013fc3dcbb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:12.070236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3991a4-9fe8-40eb-a050-4f013fc3dcbb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:12.070512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3991a4-9fe8-40eb-a050-4f013fc3dcbb None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:12.071170 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3991a4-9fe8-40eb-a050-4f013fc3dcbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:12.071471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3991a4-9fe8-40eb-a050-4f013fc3dcbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:12.072846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3991a4-9fe8-40eb-a050-4f013fc3dcbb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:12.097200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 520/1024] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:56:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:12.099258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d055fb-4fc5-4555-a11b-111d4165b793 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:12.247692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d055fb-4fc5-4555-a11b-111d4165b793 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:12.278162 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:12.290849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68219518-ec0c-452f-93a3-ec8e2d29b5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:12.291590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68219518-ec0c-452f-93a3-ec8e2d29b5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:12.291896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68219518-ec0c-452f-93a3-ec8e2d29b5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:12.292175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68219518-ec0c-452f-93a3-ec8e2d29b5c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:12.303702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68219518-ec0c-452f-93a3-ec8e2d29b5c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:12.306115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68219518-ec0c-452f-93a3-ec8e2d29b5c9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-68219518-ec0c-452f-93a3-ec8e2d29b5c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:56:12.326184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:12.327242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:12.328213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:12.328213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:12.328213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:56:12.328213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:56:12.328213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:56:12.328213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:56:12.328213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:56:12.328213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:56:12.328213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:56:12.328213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:12.330153 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 505/1025] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Aug 19 20:56:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:12.333374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3991a4-9fe8-40eb-a050-4f013fc3dcbb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:12.335351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68219518-ec0c-452f-93a3-ec8e2d29b5c9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:12.453165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3991a4-9fe8-40eb-a050-4f013fc3dcbb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:12.466987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68219518-ec0c-452f-93a3-ec8e2d29b5c9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:12.499760 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c516a-8e56-4aff-867a-26910d283307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:12.500425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c516a-8e56-4aff-867a-26910d283307 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:12.500728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c516a-8e56-4aff-867a-26910d283307 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:12.501016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c516a-8e56-4aff-867a-26910d283307 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:12.501882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c516a-8e56-4aff-867a-26910d283307 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:12.502232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c516a-8e56-4aff-867a-26910d283307 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:12.503882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c516a-8e56-4aff-867a-26910d283307 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:12.507427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:12.527349 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8de03-89c6-46d7-bec7-5528a94c943c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:12.528956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8de03-89c6-46d7-bec7-5528a94c943c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:12.529824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8de03-89c6-46d7-bec7-5528a94c943c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:12.530075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 521/1026] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Aug 19 20:56:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:12.530169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae8de03-89c6-46d7-bec7-5528a94c943c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:12.533807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8de03-89c6-46d7-bec7-5528a94c943c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:12.534419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae8de03-89c6-46d7-bec7-5528a94c943c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:12.536346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c516a-8e56-4aff-867a-26910d283307 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:12.537398 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 506/1027] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:12.546693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8de03-89c6-46d7-bec7-5528a94c943c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:12.643380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:12.687430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c516a-8e56-4aff-867a-26910d283307 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:12.713363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae8de03-89c6-46d7-bec7-5528a94c943c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:12.735925 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f77ba-d011-4339-b2cc-6a8d8fabaa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:12.736714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f77ba-d011-4339-b2cc-6a8d8fabaa71 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:12.737129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f77ba-d011-4339-b2cc-6a8d8fabaa71 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:12.737521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f77ba-d011-4339-b2cc-6a8d8fabaa71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:12.772480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17652a15-efd0-4631-b366-c49b1d5931c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:12.773244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17652a15-efd0-4631-b366-c49b1d5931c5 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:12.773528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17652a15-efd0-4631-b366-c49b1d5931c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:12.773914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17652a15-efd0-4631-b366-c49b1d5931c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:12.774635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17652a15-efd0-4631-b366-c49b1d5931c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:12.776161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17652a15-efd0-4631-b366-c49b1d5931c5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:12.815631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 507/1028] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:56:12 2022] POST /identity/v3/projects => generated 311 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:56:12.832514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:12.847895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f77ba-d011-4339-b2cc-6a8d8fabaa71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:12.850892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50f77ba-d011-4339-b2cc-6a8d8fabaa71 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:13.038249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 522/1029] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:12 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 20:56:13.044595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17652a15-efd0-4631-b366-c49b1d5931c5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:13.189412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17652a15-efd0-4631-b366-c49b1d5931c5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:13.200283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:13.244050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941afc65-7d80-4c1d-89f3-4f578fc15b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:13.244963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941afc65-7d80-4c1d-89f3-4f578fc15b52 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:13.245242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941afc65-7d80-4c1d-89f3-4f578fc15b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:13.245636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941afc65-7d80-4c1d-89f3-4f578fc15b52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:13.273667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941afc65-7d80-4c1d-89f3-4f578fc15b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:13.276396 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941afc65-7d80-4c1d-89f3-4f578fc15b52 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-941afc65-7d80-4c1d-89f3-4f578fc15b52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:13.300795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:13.302238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:56:13.303197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:13.303197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:13.303197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:13.305907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 508/1030] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:13.308407 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f77ba-d011-4339-b2cc-6a8d8fabaa71 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:13.444747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:13.456989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f77ba-d011-4339-b2cc-6a8d8fabaa71 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:13.503102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f94653-70f5-457b-90ff-287a148db214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:13.503970 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94653-70f5-457b-90ff-287a148db214 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:13.504402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94653-70f5-457b-90ff-287a148db214 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:13.504826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f94653-70f5-457b-90ff-287a148db214 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:13.505659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f94653-70f5-457b-90ff-287a148db214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:13.506164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f94653-70f5-457b-90ff-287a148db214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:13.507779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f94653-70f5-457b-90ff-287a148db214 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:13.522590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 523/1031] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:13.530037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941afc65-7d80-4c1d-89f3-4f578fc15b52 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:13.622228 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cbe32-7748-4ac8-b368-41a4892b53d1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:13.622555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cbe32-7748-4ac8-b368-41a4892b53d1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:13.622758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4cbe32-7748-4ac8-b368-41a4892b53d1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:13.624371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 524/1032] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:56:13.629891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c736389-731d-45c0-9b77-0e585d161a4b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:13.630211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c736389-731d-45c0-9b77-0e585d161a4b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:13.630462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c736389-731d-45c0-9b77-0e585d161a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:13.650774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4c736389-731d-45c0-9b77-0e585d161a4b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:13.652118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4c736389-731d-45c0-9b77-0e585d161a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:13.676715 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941afc65-7d80-4c1d-89f3-4f578fc15b52 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:13.729925 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374dd8c9-408c-431f-b3ed-ce76ccc76e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:13.730920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374dd8c9-408c-431f-b3ed-ce76ccc76e4a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:13.731996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374dd8c9-408c-431f-b3ed-ce76ccc76e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:13.732283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374dd8c9-408c-431f-b3ed-ce76ccc76e4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:13.736300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 525/1033] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:13.740340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374dd8c9-408c-431f-b3ed-ce76ccc76e4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:13.741841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374dd8c9-408c-431f-b3ed-ce76ccc76e4a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-374dd8c9-408c-431f-b3ed-ce76ccc76e4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:56:13.754689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:13.762850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:13.764730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:56:13.764730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:56:13.764730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:56:13.764730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:56:13.764730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:56:13.764730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:56:13.764730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:56:13.764730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:13.764730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 509/1034] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:56:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:13.765362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c736389-731d-45c0-9b77-0e585d161a4b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:13.872762 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:13.895285 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c736389-731d-45c0-9b77-0e585d161a4b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:13.935829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57005572-9acb-422d-8f7d-d624cc157204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:13.936447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57005572-9acb-422d-8f7d-d624cc157204 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:13.936730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57005572-9acb-422d-8f7d-d624cc157204 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:13.937005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57005572-9acb-422d-8f7d-d624cc157204 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:13.937717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57005572-9acb-422d-8f7d-d624cc157204 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:13.938054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57005572-9acb-422d-8f7d-d624cc157204 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:13.939504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57005572-9acb-422d-8f7d-d624cc157204 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:13.957811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 526/1035] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:56:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:13.963661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374dd8c9-408c-431f-b3ed-ce76ccc76e4a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:13.991865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:14.017371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57005572-9acb-422d-8f7d-d624cc157204 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:14.127157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374dd8c9-408c-431f-b3ed-ce76ccc76e4a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:14.158555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57005572-9acb-422d-8f7d-d624cc157204 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:14.176732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8c64f-20de-451e-b543-24b5e5236062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:14.177388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8c64f-20de-451e-b543-24b5e5236062 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:14.177728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8c64f-20de-451e-b543-24b5e5236062 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:14.178188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8c64f-20de-451e-b543-24b5e5236062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:14.178997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8c64f-20de-451e-b543-24b5e5236062 None admin] RBAC: Adding query filter params (role.id=a850548ca8b0447bba66f8d63c407705, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=43a9ad5341b54c40a5427045660363c3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:14.179358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8c64f-20de-451e-b543-24b5e5236062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:14.181512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8c64f-20de-451e-b543-24b5e5236062 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:14.197803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 510/1036] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:56:13 2022] GET /identity/v3/role_assignments?user.id=43a9ad5341b54c40a5427045660363c3&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a850548ca8b0447bba66f8d63c407705 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:14.219511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecd2a3-9624-4feb-af51-dc4672708ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:14.220179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecd2a3-9624-4feb-af51-dc4672708ecf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:14.221821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecd2a3-9624-4feb-af51-dc4672708ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:14.221821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ecd2a3-9624-4feb-af51-dc4672708ecf None admin] PATH_INFO: `/v3/services/image` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:14.221821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecd2a3-9624-4feb-af51-dc4672708ecf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:14.222861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ecd2a3-9624-4feb-af51-dc4672708ecf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c6ecd2a3-9624-4feb-af51-dc4672708ecf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:14.234543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:14.235639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 20:56:14.236486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:14.241862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 527/1037] 10.222.0.16 () {56 vars in 1105 bytes} [Fri Aug 19 20:56:14 2022] GET /identity/v3/services/image => generated 86 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:14.243080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8c64f-20de-451e-b543-24b5e5236062 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:14.375017 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:14.405123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8c64f-20de-451e-b543-24b5e5236062 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:14.446966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaf67b-ad7f-461d-aa90-b2c54a699c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:14.447999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaf67b-ad7f-461d-aa90-b2c54a699c1f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:14.448486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaf67b-ad7f-461d-aa90-b2c54a699c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:14.449061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aaf67b-ad7f-461d-aa90-b2c54a699c1f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:14.450238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaf67b-ad7f-461d-aa90-b2c54a699c1f None admin] RBAC: Adding query filter params (name=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:14.450845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaf67b-ad7f-461d-aa90-b2c54a699c1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:14.452830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aaf67b-ad7f-461d-aa90-b2c54a699c1f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:14.459424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 511/1038] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:14.468301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecd2a3-9624-4feb-af51-dc4672708ecf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:14.482537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:14.592976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ecd2a3-9624-4feb-af51-dc4672708ecf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:14.641050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f123059-2fe8-442d-946e-15ffd403b248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:14.642091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f123059-2fe8-442d-946e-15ffd403b248 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:14.642582 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f123059-2fe8-442d-946e-15ffd403b248 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:14.643078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f123059-2fe8-442d-946e-15ffd403b248 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:14.643960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f123059-2fe8-442d-946e-15ffd403b248 None admin] RBAC: Adding query filter params (type=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:14.644426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f123059-2fe8-442d-946e-15ffd403b248 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:14.646245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f123059-2fe8-442d-946e-15ffd403b248 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:14.660526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaf67b-ad7f-461d-aa90-b2c54a699c1f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:14.664955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 528/1039] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:14.819732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aaf67b-ad7f-461d-aa90-b2c54a699c1f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:14.871642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9934eab9-8a5b-4464-9af0-7a6d5260365a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:14.872969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934eab9-8a5b-4464-9af0-7a6d5260365a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:14.873444 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934eab9-8a5b-4464-9af0-7a6d5260365a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:14.873858 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934eab9-8a5b-4464-9af0-7a6d5260365a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:14.874636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934eab9-8a5b-4464-9af0-7a6d5260365a None admin] RBAC: Adding query filter params (interface=public, service_id=721f52df31b64ee1908e6358140d21b7, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:14.874995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934eab9-8a5b-4464-9af0-7a6d5260365a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:14.877345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9934eab9-8a5b-4464-9af0-7a6d5260365a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:14.889731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 512/1040] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Aug 19 20:56:14 2022] GET /identity/v3/endpoints?service_id=721f52df31b64ee1908e6358140d21b7&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:14.891880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f123059-2fe8-442d-946e-15ffd403b248 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:14.998838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:14.999384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c47c-a407-466c-b859-950b3d808afb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:14.999975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c47c-a407-466c-b859-950b3d808afb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:14.999975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c47c-a407-466c-b859-950b3d808afb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:15.001288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 513/1041] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:15.006821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553b817-9b39-469b-aa17-47a5b20b8c7a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:15.007204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553b817-9b39-469b-aa17-47a5b20b8c7a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:15.007395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553b817-9b39-469b-aa17-47a5b20b8c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:15.026653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9553b817-9b39-469b-aa17-47a5b20b8c7a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:15.027896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9553b817-9b39-469b-aa17-47a5b20b8c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:15.031149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:15.043871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f123059-2fe8-442d-946e-15ffd403b248 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:15.096400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62290b-f8a7-4c03-9f80-5fa0a0b0c56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:15.097061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62290b-f8a7-4c03-9f80-5fa0a0b0c56d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:15.097447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62290b-f8a7-4c03-9f80-5fa0a0b0c56d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:15.097702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62290b-f8a7-4c03-9f80-5fa0a0b0c56d None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:15.098386 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62290b-f8a7-4c03-9f80-5fa0a0b0c56d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:15.099975 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62290b-f8a7-4c03-9f80-5fa0a0b0c56d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:15.108730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 529/1042] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Aug 19 20:56:14 2022] GET /identity/v3/services => generated 1957 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 20:56:15.135281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 514/1043] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:15.441826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62290b-f8a7-4c03-9f80-5fa0a0b0c56d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:15.577601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62290b-f8a7-4c03-9f80-5fa0a0b0c56d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:15.619905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29900ea8-7ffc-4fa6-95f9-e672479f0ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:15.620922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29900ea8-7ffc-4fa6-95f9-e672479f0ebd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:15.621508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29900ea8-7ffc-4fa6-95f9-e672479f0ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:15.622720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29900ea8-7ffc-4fa6-95f9-e672479f0ebd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:15.624004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29900ea8-7ffc-4fa6-95f9-e672479f0ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:15.625496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29900ea8-7ffc-4fa6-95f9-e672479f0ebd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:15.627343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 530/1044] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:15.633241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553b817-9b39-469b-aa17-47a5b20b8c7a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:15.739884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553b817-9b39-469b-aa17-47a5b20b8c7a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:15.778095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa922c5e-2bea-48a5-9c40-f60c994db8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:15.778732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa922c5e-2bea-48a5-9c40-f60c994db8a0 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:15.779135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa922c5e-2bea-48a5-9c40-f60c994db8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:15.779382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa922c5e-2bea-48a5-9c40-f60c994db8a0 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:15.780049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa922c5e-2bea-48a5-9c40-f60c994db8a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:15.781517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa922c5e-2bea-48a5-9c40-f60c994db8a0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:15.793328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:15.821793 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 515/1045] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:56:15 2022] POST /identity/v3/users => generated 285 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:56:16.310652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fab0e1-a596-4603-bcdc-eb124c938508 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:16.311064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fab0e1-a596-4603-bcdc-eb124c938508 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:16.311241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fab0e1-a596-4603-bcdc-eb124c938508 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:16.312653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 531/1046] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:16.321330 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c0704-105a-48f9-aadd-2c97d7a2c0c7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:16.321932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c0704-105a-48f9-aadd-2c97d7a2c0c7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:16.322353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c0704-105a-48f9-aadd-2c97d7a2c0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:16.353621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8e4c0704-105a-48f9-aadd-2c97d7a2c0c7 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:16.355649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8e4c0704-105a-48f9-aadd-2c97d7a2c0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:16.430275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 516/1047] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:16.919648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fab0e1-a596-4603-bcdc-eb124c938508 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:16.925886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:17.034654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fab0e1-a596-4603-bcdc-eb124c938508 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:17.072818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e645a75-4b49-4d3d-9079-f1ee9ad0d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:17.073459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e645a75-4b49-4d3d-9079-f1ee9ad0d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:17.073739 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e645a75-4b49-4d3d-9079-f1ee9ad0d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:17.074055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e645a75-4b49-4d3d-9079-f1ee9ad0d8c4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:17.094283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e645a75-4b49-4d3d-9079-f1ee9ad0d8c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6e645a75-4b49-4d3d-9079-f1ee9ad0d8c4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:56:17.099141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:17.100053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:17.100924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 532/1048] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Aug 19 20:56:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:17.108232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c0704-105a-48f9-aadd-2c97d7a2c0c7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:17.216173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c0704-105a-48f9-aadd-2c97d7a2c0c7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:17.252171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:17.261975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b707ce-e9d1-4102-af0d-c62df30dd0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:17.263246 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b707ce-e9d1-4102-af0d-c62df30dd0bf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:17.263543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b707ce-e9d1-4102-af0d-c62df30dd0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:17.263806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b707ce-e9d1-4102-af0d-c62df30dd0bf None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:17.264484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b707ce-e9d1-4102-af0d-c62df30dd0bf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:17.264788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b707ce-e9d1-4102-af0d-c62df30dd0bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:17.266194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b707ce-e9d1-4102-af0d-c62df30dd0bf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:17.280462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 517/1049] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Aug 19 20:56:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:17.282117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e645a75-4b49-4d3d-9079-f1ee9ad0d8c4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:17.350793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:17.394305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e645a75-4b49-4d3d-9079-f1ee9ad0d8c4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:17.401748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a55b53-f7c8-493b-bd47-e79b692d949e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:17.402309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a55b53-f7c8-493b-bd47-e79b692d949e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:17.402309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a55b53-f7c8-493b-bd47-e79b692d949e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:17.404204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 518/1050] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:17.409937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2cc67-dfcb-4f11-9f68-7087cc07d775 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:17.410244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2cc67-dfcb-4f11-9f68-7087cc07d775 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:17.410497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c2cc67-dfcb-4f11-9f68-7087cc07d775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:17.428872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-70c2cc67-dfcb-4f11-9f68-7087cc07d775 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:17.430157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-70c2cc67-dfcb-4f11-9f68-7087cc07d775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:17.435471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb5ee4-638e-44bf-8187-157dbea2428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:17.436113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb5ee4-638e-44bf-8187-157dbea2428e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:17.436389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb5ee4-638e-44bf-8187-157dbea2428e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:17.436657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb5ee4-638e-44bf-8187-157dbea2428e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:17.447794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb5ee4-638e-44bf-8187-157dbea2428e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:17.449263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb5ee4-638e-44bf-8187-157dbea2428e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-babb5ee4-638e-44bf-8187-157dbea2428e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:17.466842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:17.468099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:56:17.469015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:17.469015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:17.469015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:17.472781 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 533/1051] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:17.473967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb5ee4-638e-44bf-8187-157dbea2428e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:17.508425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 519/1052] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:17 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:17.593001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb5ee4-638e-44bf-8187-157dbea2428e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:17.610224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:17.630078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfcab5-5636-4d98-9c83-76d8e35f6838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:17.630689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfcab5-5636-4d98-9c83-76d8e35f6838 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:17.630962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfcab5-5636-4d98-9c83-76d8e35f6838 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:17.631231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adfcab5-5636-4d98-9c83-76d8e35f6838 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:17.631872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adfcab5-5636-4d98-9c83-76d8e35f6838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:17.632168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adfcab5-5636-4d98-9c83-76d8e35f6838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:17.633527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adfcab5-5636-4d98-9c83-76d8e35f6838 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:17.645923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 534/1053] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:17.651801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2cc67-dfcb-4f11-9f68-7087cc07d775 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:17.763081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c2cc67-dfcb-4f11-9f68-7087cc07d775 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:17.781174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfcab5-5636-4d98-9c83-76d8e35f6838 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:17.782152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:17.804841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c42ff-58d0-42d0-a7c0-d4d1eb7af03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:17.805504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c42ff-58d0-42d0-a7c0-d4d1eb7af03f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:17.805843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c42ff-58d0-42d0-a7c0-d4d1eb7af03f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:17.806116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c42ff-58d0-42d0-a7c0-d4d1eb7af03f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:17.815589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c42ff-58d0-42d0-a7c0-d4d1eb7af03f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:17.817046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c42ff-58d0-42d0-a7c0-d4d1eb7af03f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-218c42ff-58d0-42d0-a7c0-d4d1eb7af03f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:56:17.829259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:17.830378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:17.831339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:17.831339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:17.831339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:56:17.831339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:56:17.831339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:56:17.831339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:56:17.831339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:56:17.831339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:56:17.831339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:56:17.831339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:17.835240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 520/1054] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Aug 19 20:56:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:17.836639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c42ff-58d0-42d0-a7c0-d4d1eb7af03f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:17.911534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adfcab5-5636-4d98-9c83-76d8e35f6838 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:17.952961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d55e50-5fb4-4545-bfe1-3beab5adb3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:17.953685 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55e50-5fb4-4545-bfe1-3beab5adb3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:17.953942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55e50-5fb4-4545-bfe1-3beab5adb3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:17.954226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d55e50-5fb4-4545-bfe1-3beab5adb3c6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:17.954725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c42ff-58d0-42d0-a7c0-d4d1eb7af03f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:17.954824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d55e50-5fb4-4545-bfe1-3beab5adb3c6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:17.957355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d55e50-5fb4-4545-bfe1-3beab5adb3c6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-36d55e50-5fb4-4545-bfe1-3beab5adb3c6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:17.970924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:17.972028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:17.972881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 535/1055] 10.222.0.16 () {56 vars in 1105 bytes} [Fri Aug 19 20:56:17 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:17.973675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34767b64-df5f-4f1b-a327-c9ca8216e91c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:17.974262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34767b64-df5f-4f1b-a327-c9ca8216e91c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:17.974262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34767b64-df5f-4f1b-a327-c9ca8216e91c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:17.975618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 536/1056] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:17.978244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34767b64-df5f-4f1b-a327-c9ca8216e91c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:17.995791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef031a94-c6ef-4d1c-a004-0694695cc9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:17.997168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031a94-c6ef-4d1c-a004-0694695cc9da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:17.997539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031a94-c6ef-4d1c-a004-0694695cc9da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:17.998001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef031a94-c6ef-4d1c-a004-0694695cc9da None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:17.998945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef031a94-c6ef-4d1c-a004-0694695cc9da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:17.999262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef031a94-c6ef-4d1c-a004-0694695cc9da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:18.000806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef031a94-c6ef-4d1c-a004-0694695cc9da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:18.027716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 521/1057] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Aug 19 20:56:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:18.027716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a78c55-ea02-4b6c-8247-b5bc83f808f6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:18.027716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a78c55-ea02-4b6c-8247-b5bc83f808f6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:18.027716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a78c55-ea02-4b6c-8247-b5bc83f808f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:18.045570 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:18.046590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-91a78c55-ea02-4b6c-8247-b5bc83f808f6 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:18.047856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-91a78c55-ea02-4b6c-8247-b5bc83f808f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:18.082511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:18.124785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34767b64-df5f-4f1b-a327-c9ca8216e91c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:18.148985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 522/1058] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:18.149932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a78c55-ea02-4b6c-8247-b5bc83f808f6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:18.186165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5e30f-a3ac-47b5-a296-63b83c5f1923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:18.187110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e30f-a3ac-47b5-a296-63b83c5f1923 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:18.187485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e30f-a3ac-47b5-a296-63b83c5f1923 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:18.188365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5e30f-a3ac-47b5-a296-63b83c5f1923 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:18.188845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5e30f-a3ac-47b5-a296-63b83c5f1923 None admin] RBAC: Adding query filter params (name=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:18.189316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5e30f-a3ac-47b5-a296-63b83c5f1923 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:18.191696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a5e30f-a3ac-47b5-a296-63b83c5f1923 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:18.197618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 537/1059] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:18.204387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5e30f-a3ac-47b5-a296-63b83c5f1923 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:18.303748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a78c55-ea02-4b6c-8247-b5bc83f808f6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:18.333284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5e30f-a3ac-47b5-a296-63b83c5f1923 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:18.336974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:18.353581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca73dc-7be3-49ee-93ef-92c1603e81b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:18.354785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca73dc-7be3-49ee-93ef-92c1603e81b9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:18.355421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca73dc-7be3-49ee-93ef-92c1603e81b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:18.355856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca73dc-7be3-49ee-93ef-92c1603e81b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:18.370724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca73dc-7be3-49ee-93ef-92c1603e81b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:18.373075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca73dc-7be3-49ee-93ef-92c1603e81b9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:18.375940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e48c4-0eba-41f7-a255-eee2b10d6253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:18.376564 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e48c4-0eba-41f7-a255-eee2b10d6253 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:18.376850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e48c4-0eba-41f7-a255-eee2b10d6253 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:18.377128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e48c4-0eba-41f7-a255-eee2b10d6253 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:18.377795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e48c4-0eba-41f7-a255-eee2b10d6253 None admin] RBAC: Adding query filter params (type=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:18.378113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e48c4-0eba-41f7-a255-eee2b10d6253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:18.379520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e48c4-0eba-41f7-a255-eee2b10d6253 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:18.391303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 538/1060] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ecca73dc-7be3-49ee-93ef-92c1603e81b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:18.392325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:18.393278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:56:18.394257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:18.394257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:18.394257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:18.394427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e48c4-0eba-41f7-a255-eee2b10d6253 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:18.399281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 523/1061] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:18.399771 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca73dc-7be3-49ee-93ef-92c1603e81b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:18.424671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:18.526384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e48c4-0eba-41f7-a255-eee2b10d6253 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:18.547445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca73dc-7be3-49ee-93ef-92c1603e81b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:18.568017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6155a-86b0-49b5-803b-e989dd52aadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:18.568546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6155a-86b0-49b5-803b-e989dd52aadf None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:18.568899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6155a-86b0-49b5-803b-e989dd52aadf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:18.569178 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6155a-86b0-49b5-803b-e989dd52aadf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:18.569964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6155a-86b0-49b5-803b-e989dd52aadf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:18.571421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6155a-86b0-49b5-803b-e989dd52aadf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:18.587200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d1ab6-382b-4850-90c2-3ddf4077ced8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:18.587813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d1ab6-382b-4850-90c2-3ddf4077ced8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:18.588102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d1ab6-382b-4850-90c2-3ddf4077ced8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:18.588371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d1ab6-382b-4850-90c2-3ddf4077ced8 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:18.589060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d1ab6-382b-4850-90c2-3ddf4077ced8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:18.589409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d1ab6-382b-4850-90c2-3ddf4077ced8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:18.591622 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d1ab6-382b-4850-90c2-3ddf4077ced8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:18.605629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 539/1062] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 20:56:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:56:18.605629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6155a-86b0-49b5-803b-e989dd52aadf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:18.613909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 524/1063] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:18.622366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d1ab6-382b-4850-90c2-3ddf4077ced8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:18.640764 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:18.749648 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:18.762182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6155a-86b0-49b5-803b-e989dd52aadf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:18.781478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d1ab6-382b-4850-90c2-3ddf4077ced8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:18.824448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c38de9-e00c-4a8a-9a41-142956df08d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:18.825153 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c38de9-e00c-4a8a-9a41-142956df08d0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:18.825429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c38de9-e00c-4a8a-9a41-142956df08d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:18.825711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c38de9-e00c-4a8a-9a41-142956df08d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:18.836815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aef35-3c52-46ff-8287-683f4ef18591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:18.837603 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aef35-3c52-46ff-8287-683f4ef18591 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:18.837603 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aef35-3c52-46ff-8287-683f4ef18591 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:18.837915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50aef35-3c52-46ff-8287-683f4ef18591 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:18.838877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c38de9-e00c-4a8a-9a41-142956df08d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-22c38de9-e00c-4a8a-9a41-142956df08d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:56:18.845315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:18.846483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:18.847773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 540/1064] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:56:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:18.849368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50aef35-3c52-46ff-8287-683f4ef18591 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:18.850963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50aef35-3c52-46ff-8287-683f4ef18591 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:18.854705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c38de9-e00c-4a8a-9a41-142956df08d0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d50aef35-3c52-46ff-8287-683f4ef18591 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:56:18.866669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:18.867754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:18.868774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:56:18.868774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:56:18.868774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:56:18.868774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:56:18.868774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:56:18.868774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:56:18.868774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:56:18.868774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:18.868774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 525/1065] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:56:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:18.878764 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aef35-3c52-46ff-8287-683f4ef18591 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:18.963315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:18.988834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c38de9-e00c-4a8a-9a41-142956df08d0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:19.011060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50aef35-3c52-46ff-8287-683f4ef18591 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:19.034305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae34e60-373e-4d59-b328-97935c970c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:19.034931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae34e60-373e-4d59-b328-97935c970c07 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:19.035262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae34e60-373e-4d59-b328-97935c970c07 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:19.035490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae34e60-373e-4d59-b328-97935c970c07 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:19.036375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae34e60-373e-4d59-b328-97935c970c07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:19.036680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae34e60-373e-4d59-b328-97935c970c07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:19.038094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae34e60-373e-4d59-b328-97935c970c07 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:19.048743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 541/1066] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:56:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:19.054678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae34e60-373e-4d59-b328-97935c970c07 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:19.063695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545d7fe-ab37-4248-bf2f-cdf31eca369a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:19.064321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545d7fe-ab37-4248-bf2f-cdf31eca369a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:19.064600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545d7fe-ab37-4248-bf2f-cdf31eca369a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:19.064872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4545d7fe-ab37-4248-bf2f-cdf31eca369a None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:19.065582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545d7fe-ab37-4248-bf2f-cdf31eca369a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:19.065926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545d7fe-ab37-4248-bf2f-cdf31eca369a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:19.067399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4545d7fe-ab37-4248-bf2f-cdf31eca369a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:19.078767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:19.093970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 526/1067] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:56:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:19.096869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545d7fe-ab37-4248-bf2f-cdf31eca369a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:19.194685 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae34e60-373e-4d59-b328-97935c970c07 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:19.229813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4545d7fe-ab37-4248-bf2f-cdf31eca369a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:19.234215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5465043-9352-4d28-892e-e1943e97e512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:19.234861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5465043-9352-4d28-892e-e1943e97e512 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:19.235142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5465043-9352-4d28-892e-e1943e97e512 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:19.235416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5465043-9352-4d28-892e-e1943e97e512 None admin] PATH_INFO: `/v3/users/70915623214646128afd2eb4dd871b92` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:19.249112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:19.252971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5465043-9352-4d28-892e-e1943e97e512 None admin] RBAC: Authorizing `identity:get_user(user_id=70915623214646128afd2eb4dd871b92)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:19.255284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5465043-9352-4d28-892e-e1943e97e512 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:19.257863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 542/1068] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:19 2022] GET /identity/v3/users/70915623214646128afd2eb4dd871b92 => generated 285 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:19.264058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5465043-9352-4d28-892e-e1943e97e512 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:19.278470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38d0cb-5a14-4156-bc3b-b4239ecda6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:19.279266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38d0cb-5a14-4156-bc3b-b4239ecda6ad None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:19.279649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38d0cb-5a14-4156-bc3b-b4239ecda6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:19.280005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38d0cb-5a14-4156-bc3b-b4239ecda6ad None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/43a9ad5341b54c40a5427045660363c3/roles/a850548ca8b0447bba66f8d63c407705` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:19.282650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38d0cb-5a14-4156-bc3b-b4239ecda6ad None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=43a9ad5341b54c40a5427045660363c3, role_id=a850548ca8b0447bba66f8d63c407705)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:19.284274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38d0cb-5a14-4156-bc3b-b4239ecda6ad None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:19.300481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 527/1069] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:56:19 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/43a9ad5341b54c40a5427045660363c3/roles/a850548ca8b0447bba66f8d63c407705 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 20:56:19.428131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5465043-9352-4d28-892e-e1943e97e512 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:19.475786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e56fe7-4790-4950-846f-7c1002a313be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:19.476412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e56fe7-4790-4950-846f-7c1002a313be None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:19.476694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e56fe7-4790-4950-846f-7c1002a313be None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:19.476968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e56fe7-4790-4950-846f-7c1002a313be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:19.563872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:19.567509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e56fe7-4790-4950-846f-7c1002a313be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:19.568990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e56fe7-4790-4950-846f-7c1002a313be None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:19.692337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 543/1070] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 20:56:19.698559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38d0cb-5a14-4156-bc3b-b4239ecda6ad None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:19.807176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:19.888164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38d0cb-5a14-4156-bc3b-b4239ecda6ad None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:19.928276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa166e37-aa7b-472d-8c37-d36c6a209a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:19.928852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166e37-aa7b-472d-8c37-d36c6a209a10 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:19.929132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166e37-aa7b-472d-8c37-d36c6a209a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:19.929408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa166e37-aa7b-472d-8c37-d36c6a209a10 None admin] PATH_INFO: `/v3/projects/cbb28a11737b4187aef8af9c06619e8a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:19.930640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa166e37-aa7b-472d-8c37-d36c6a209a10 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb28a11737b4187aef8af9c06619e8a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:19.932084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa166e37-aa7b-472d-8c37-d36c6a209a10 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:19.934190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 528/1071] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:56:19 2022] GET /identity/v3/projects/cbb28a11737b4187aef8af9c06619e8a => generated 311 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:19.940900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e56fe7-4790-4950-846f-7c1002a313be None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:20.113714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:20.118184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e56fe7-4790-4950-846f-7c1002a313be None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:20.172395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff6236-edad-4f3f-a399-55136e251f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:20.173180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff6236-edad-4f3f-a399-55136e251f8a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:20.173554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff6236-edad-4f3f-a399-55136e251f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:20.173934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff6236-edad-4f3f-a399-55136e251f8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:20.174784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff6236-edad-4f3f-a399-55136e251f8a None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=cbb28a11737b4187aef8af9c06619e8a, user.id=70915623214646128afd2eb4dd871b92) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:20.175094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff6236-edad-4f3f-a399-55136e251f8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:20.177004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff6236-edad-4f3f-a399-55136e251f8a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:20.186159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 544/1072] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:56:19 2022] GET /identity/v3/role_assignments?user.id=70915623214646128afd2eb4dd871b92&scope.project.id=cbb28a11737b4187aef8af9c06619e8a&role.id=8b350845f331423cb0494e3449b58310 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:20.691713 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be03276-4180-47cc-8e1d-2dc5cec433ab None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:20.692069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be03276-4180-47cc-8e1d-2dc5cec433ab None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:20.692262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be03276-4180-47cc-8e1d-2dc5cec433ab None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:20.693842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 529/1073] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:20.699370 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14def7d1-16ae-4e51-84b9-ff876e4f4c1f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:20.700341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14def7d1-16ae-4e51-84b9-ff876e4f4c1f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:20.700512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14def7d1-16ae-4e51-84b9-ff876e4f4c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:20.720252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-14def7d1-16ae-4e51-84b9-ff876e4f4c1f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:20.721524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-14def7d1-16ae-4e51-84b9-ff876e4f4c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:20.813684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 545/1074] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:21.077676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be03276-4180-47cc-8e1d-2dc5cec433ab None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:21.197143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be03276-4180-47cc-8e1d-2dc5cec433ab None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:21.225975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:21.235886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301f3f3-eab8-4c91-a9e1-6dd4bb282a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:21.236501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f3f3-eab8-4c91-a9e1-6dd4bb282a09 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:21.236777 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f3f3-eab8-4c91-a9e1-6dd4bb282a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:21.237048 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301f3f3-eab8-4c91-a9e1-6dd4bb282a09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:21.310449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301f3f3-eab8-4c91-a9e1-6dd4bb282a09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:21.311965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c301f3f3-eab8-4c91-a9e1-6dd4bb282a09 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:21.312856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1ff5e-f3e6-4ac5-9980-7441876d72db None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:21.313711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1ff5e-f3e6-4ac5-9980-7441876d72db None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:21.314471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1ff5e-f3e6-4ac5-9980-7441876d72db None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:21.317416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 546/1075] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:21.322599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913b21d-63e5-494d-b9a5-66cdcf1524b6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:21.322887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913b21d-63e5-494d-b9a5-66cdcf1524b6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:21.323361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913b21d-63e5-494d-b9a5-66cdcf1524b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:21.342748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e913b21d-63e5-494d-b9a5-66cdcf1524b6 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:21.343968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e913b21d-63e5-494d-b9a5-66cdcf1524b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:21.421053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 547/1076] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:21.425918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 530/1077] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:56:21.431406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913b21d-63e5-494d-b9a5-66cdcf1524b6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:21.459148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:21.540031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913b21d-63e5-494d-b9a5-66cdcf1524b6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:21.574477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20cd78-3a1b-4a76-8870-26e96e89a4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:21.576077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20cd78-3a1b-4a76-8870-26e96e89a4bd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:21.576552 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20cd78-3a1b-4a76-8870-26e96e89a4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:21.577050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c20cd78-3a1b-4a76-8870-26e96e89a4bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:21.587680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20cd78-3a1b-4a76-8870-26e96e89a4bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:21.589358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c20cd78-3a1b-4a76-8870-26e96e89a4bd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1c20cd78-3a1b-4a76-8870-26e96e89a4bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:21.605140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:21.606779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:56:21.607707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:21.607707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:21.607707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:21.609184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 548/1078] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:21.615242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301f3f3-eab8-4c91-a9e1-6dd4bb282a09 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:21.643599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:21.695365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20cd78-3a1b-4a76-8870-26e96e89a4bd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:21.726589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301f3f3-eab8-4c91-a9e1-6dd4bb282a09 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:21.770797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f59af-d2e2-4ae0-b1cd-5ed051fd6a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:21.771557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f59af-d2e2-4ae0-b1cd-5ed051fd6a89 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:21.771840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f59af-d2e2-4ae0-b1cd-5ed051fd6a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:21.772117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f59af-d2e2-4ae0-b1cd-5ed051fd6a89 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:21.772801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f59af-d2e2-4ae0-b1cd-5ed051fd6a89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:21.773112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f59af-d2e2-4ae0-b1cd-5ed051fd6a89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:21.774571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f59af-d2e2-4ae0-b1cd-5ed051fd6a89 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:21.784983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:21.798005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 531/1079] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:21.814151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c20cd78-3a1b-4a76-8870-26e96e89a4bd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:21.855233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb344c6-ab6a-4b48-b8dc-8bfb9a030135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:21.855792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb344c6-ab6a-4b48-b8dc-8bfb9a030135 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:21.856727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb344c6-ab6a-4b48-b8dc-8bfb9a030135 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:21.856936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb344c6-ab6a-4b48-b8dc-8bfb9a030135 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:21.868642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb344c6-ab6a-4b48-b8dc-8bfb9a030135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2eb344c6-ab6a-4b48-b8dc-8bfb9a030135 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:56:21.874321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:21.875311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:21.878745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 549/1080] 10.222.0.16 () {56 vars in 1120 bytes} [Fri Aug 19 20:56:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:21.881708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f59af-d2e2-4ae0-b1cd-5ed051fd6a89 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:21.978147 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:21.995267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f59af-d2e2-4ae0-b1cd-5ed051fd6a89 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:22.033066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd59fe-3577-4457-b2dc-a61658d8e416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:22.033066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd59fe-3577-4457-b2dc-a61658d8e416 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:22.033367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd59fe-3577-4457-b2dc-a61658d8e416 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:22.033619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd59fe-3577-4457-b2dc-a61658d8e416 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:22.034386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd59fe-3577-4457-b2dc-a61658d8e416 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:22.034696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd59fe-3577-4457-b2dc-a61658d8e416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:22.036085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd59fe-3577-4457-b2dc-a61658d8e416 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:22.046478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 532/1081] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Aug 19 20:56:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:22.052481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb344c6-ab6a-4b48-b8dc-8bfb9a030135 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:22.165814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb344c6-ab6a-4b48-b8dc-8bfb9a030135 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:22.177311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9eed3-d72c-4d56-8387-2a5f82a2ad98 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:22.177866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9eed3-d72c-4d56-8387-2a5f82a2ad98 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:22.177971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9eed3-d72c-4d56-8387-2a5f82a2ad98 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:22.179257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 533/1082] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:56:22.185195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9eb36-f9a3-4d9f-813f-f2368288247c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:22.185532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9eb36-f9a3-4d9f-813f-f2368288247c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:22.185781 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e9eb36-f9a3-4d9f-813f-f2368288247c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:22.201086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47efd3-22f8-4b8a-a3ac-bf2cd0175a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:22.201719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47efd3-22f8-4b8a-a3ac-bf2cd0175a1f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:22.202028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47efd3-22f8-4b8a-a3ac-bf2cd0175a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:22.202298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47efd3-22f8-4b8a-a3ac-bf2cd0175a1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:22.204079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d6e9eb36-f9a3-4d9f-813f-f2368288247c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:22.204984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:22.205298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d6e9eb36-f9a3-4d9f-813f-f2368288247c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:22.213720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47efd3-22f8-4b8a-a3ac-bf2cd0175a1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:22.215269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47efd3-22f8-4b8a-a3ac-bf2cd0175a1f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2d47efd3-22f8-4b8a-a3ac-bf2cd0175a1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:22.229827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:22.231491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:56:22.232520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:22.232520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:22.232520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:22.232520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 550/1083] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:22.237719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47efd3-22f8-4b8a-a3ac-bf2cd0175a1f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:22.277191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 534/1084] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:22.337257 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47efd3-22f8-4b8a-a3ac-bf2cd0175a1f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:22.370884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af135acd-36ee-4b4c-9a00-3fd270bb960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:22.372042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af135acd-36ee-4b4c-9a00-3fd270bb960d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:22.372318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af135acd-36ee-4b4c-9a00-3fd270bb960d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:22.372587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af135acd-36ee-4b4c-9a00-3fd270bb960d None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:22.373348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af135acd-36ee-4b4c-9a00-3fd270bb960d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:22.373649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af135acd-36ee-4b4c-9a00-3fd270bb960d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:22.375033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af135acd-36ee-4b4c-9a00-3fd270bb960d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:22.386497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 551/1085] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:22.392938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9eb36-f9a3-4d9f-813f-f2368288247c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:22.470474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:22.492736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e9eb36-f9a3-4d9f-813f-f2368288247c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:22.528575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc20a59-cd16-449c-b7af-e5123b9a4d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:22.529198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc20a59-cd16-449c-b7af-e5123b9a4d17 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:22.529476 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc20a59-cd16-449c-b7af-e5123b9a4d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:22.529750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc20a59-cd16-449c-b7af-e5123b9a4d17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:22.536737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc20a59-cd16-449c-b7af-e5123b9a4d17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:22.538213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc20a59-cd16-449c-b7af-e5123b9a4d17 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:22.548236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af135acd-36ee-4b4c-9a00-3fd270bb960d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5cc20a59-cd16-449c-b7af-e5123b9a4d17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:56:22.549628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:22.550760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:22.551799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:22.551799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:22.551799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:56:22.551799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:56:22.551799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:56:22.551799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:56:22.551799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:56:22.551799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:56:22.551799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 19 20:56:22.551799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:22.553919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 535/1086] 10.222.0.16 () {56 vars in 1093 bytes} [Fri Aug 19 20:56:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:22.556362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc20a59-cd16-449c-b7af-e5123b9a4d17 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:22.598149 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:22.652048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af135acd-36ee-4b4c-9a00-3fd270bb960d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:22.658546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc20a59-cd16-449c-b7af-e5123b9a4d17 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:22.687607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495be2d-03bf-4f25-8c92-d71a483d967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:22.688233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495be2d-03bf-4f25-8c92-d71a483d967f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:22.688509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495be2d-03bf-4f25-8c92-d71a483d967f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:22.688781 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e495be2d-03bf-4f25-8c92-d71a483d967f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:22.692945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fca6fc-259d-4a94-bdb7-8e432da534a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:22.693552 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fca6fc-259d-4a94-bdb7-8e432da534a5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:22.693861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fca6fc-259d-4a94-bdb7-8e432da534a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:22.694136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fca6fc-259d-4a94-bdb7-8e432da534a5 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:22.694866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fca6fc-259d-4a94-bdb7-8e432da534a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:22.695171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fca6fc-259d-4a94-bdb7-8e432da534a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:22.696604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fca6fc-259d-4a94-bdb7-8e432da534a5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:22.698945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e495be2d-03bf-4f25-8c92-d71a483d967f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e495be2d-03bf-4f25-8c92-d71a483d967f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:56:22.704423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:22.705555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:22.707423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 552/1087] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:56:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:22.708655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:22.711667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495be2d-03bf-4f25-8c92-d71a483d967f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:22.719140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 536/1088] 10.222.0.16 () {56 vars in 1133 bytes} [Fri Aug 19 20:56:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:22.725133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fca6fc-259d-4a94-bdb7-8e432da534a5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:22.822687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e495be2d-03bf-4f25-8c92-d71a483d967f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:22.832711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:22.836113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fca6fc-259d-4a94-bdb7-8e432da534a5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:22.858713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21735863-9edd-468c-844c-0f045b71546f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:22.859322 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735863-9edd-468c-844c-0f045b71546f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:22.859607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735863-9edd-468c-844c-0f045b71546f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:22.859874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21735863-9edd-468c-844c-0f045b71546f None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:22.860542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21735863-9edd-468c-844c-0f045b71546f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:22.860841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21735863-9edd-468c-844c-0f045b71546f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:22.862249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21735863-9edd-468c-844c-0f045b71546f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:22.870784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 553/1089] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:56:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:22.870950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ac557-3b63-4a1d-9a50-3a8229e663f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:22.871559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ac557-3b63-4a1d-9a50-3a8229e663f3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:22.871854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ac557-3b63-4a1d-9a50-3a8229e663f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:22.872095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ac557-3b63-4a1d-9a50-3a8229e663f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:22.876609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21735863-9edd-468c-844c-0f045b71546f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:22.936015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:22.942910 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ac557-3b63-4a1d-9a50-3a8229e663f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:22.944421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ac557-3b63-4a1d-9a50-3a8229e663f3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:22.977798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21735863-9edd-468c-844c-0f045b71546f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:23.013176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf430ce-0670-4633-9296-09a5a5543c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:23.014005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf430ce-0670-4633-9296-09a5a5543c2b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:23.014324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf430ce-0670-4633-9296-09a5a5543c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:23.014593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf430ce-0670-4633-9296-09a5a5543c2b None admin] PATH_INFO: `/v3/users/70915623214646128afd2eb4dd871b92` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:23.015763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf430ce-0670-4633-9296-09a5a5543c2b None admin] RBAC: Authorizing `identity:get_user(user_id=70915623214646128afd2eb4dd871b92)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:23.017207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf430ce-0670-4633-9296-09a5a5543c2b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:23.018950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 554/1090] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:22 2022] GET /identity/v3/users/70915623214646128afd2eb4dd871b92 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:23.024670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf430ce-0670-4633-9296-09a5a5543c2b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:23.048753 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 537/1091] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 20:56:23.054274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ac557-3b63-4a1d-9a50-3a8229e663f3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:23.067624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:23.132913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf430ce-0670-4633-9296-09a5a5543c2b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:23.166531 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:23.174152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99294b37-11c9-4550-9dca-812203be5e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:23.174787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99294b37-11c9-4550-9dca-812203be5e94 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:23.175063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99294b37-11c9-4550-9dca-812203be5e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:23.175183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ac557-3b63-4a1d-9a50-3a8229e663f3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:23.175333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99294b37-11c9-4550-9dca-812203be5e94 None admin] PATH_INFO: `/v3/projects/cbb28a11737b4187aef8af9c06619e8a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:23.176456 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99294b37-11c9-4550-9dca-812203be5e94 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb28a11737b4187aef8af9c06619e8a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:23.177913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99294b37-11c9-4550-9dca-812203be5e94 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:23.179581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 555/1092] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:56:23 2022] GET /identity/v3/projects/cbb28a11737b4187aef8af9c06619e8a => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:23.186127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99294b37-11c9-4550-9dca-812203be5e94 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:23.213577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a334f-f0fc-468c-b833-fd26c281d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:23.214241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a334f-f0fc-468c-b833-fd26c281d1bc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:23.214514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a334f-f0fc-468c-b833-fd26c281d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:23.214815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a334f-f0fc-468c-b833-fd26c281d1bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:23.226028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a334f-f0fc-468c-b833-fd26c281d1bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:23.227489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a334f-f0fc-468c-b833-fd26c281d1bc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-dc0a334f-f0fc-468c-b833-fd26c281d1bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:23.243808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:23.245078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:56:23.245986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:23.245986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:56:23.245986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:23.245986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 538/1093] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:56:23.251395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a334f-f0fc-468c-b833-fd26c281d1bc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:23.291097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99294b37-11c9-4550-9dca-812203be5e94 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:23.298687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:23.327388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974747f3-2b75-4e85-bafc-8b504355eafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:23.328039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974747f3-2b75-4e85-bafc-8b504355eafa None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:23.328325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974747f3-2b75-4e85-bafc-8b504355eafa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:23.328598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974747f3-2b75-4e85-bafc-8b504355eafa None admin] PATH_INFO: `/v3/projects/cbb28a11737b4187aef8af9c06619e8a/users/70915623214646128afd2eb4dd871b92/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:23.330523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974747f3-2b75-4e85-bafc-8b504355eafa None admin] RBAC: Authorizing `identity:create_grant(project_id=cbb28a11737b4187aef8af9c06619e8a, user_id=70915623214646128afd2eb4dd871b92, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:23.331955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974747f3-2b75-4e85-bafc-8b504355eafa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:23.346154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 556/1094] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:56:23 2022] PUT /identity/v3/projects/cbb28a11737b4187aef8af9c06619e8a/users/70915623214646128afd2eb4dd871b92/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 20:56:23.354337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a334f-f0fc-468c-b833-fd26c281d1bc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:23.398277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9f85-af93-4f18-b530-6dbaa91b9010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:23.398917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9f85-af93-4f18-b530-6dbaa91b9010 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:23.399208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9f85-af93-4f18-b530-6dbaa91b9010 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:23.399485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba9f85-af93-4f18-b530-6dbaa91b9010 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:23.400157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9f85-af93-4f18-b530-6dbaa91b9010 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:23.400463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9f85-af93-4f18-b530-6dbaa91b9010 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:23.402025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba9f85-af93-4f18-b530-6dbaa91b9010 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:23.416462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 539/1095] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:23.422587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974747f3-2b75-4e85-bafc-8b504355eafa None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:23.544648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974747f3-2b75-4e85-bafc-8b504355eafa None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:23.583014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af89a-ba7e-47f6-80a6-ae01bcd89646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:23.583666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af89a-ba7e-47f6-80a6-ae01bcd89646 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:23.583946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af89a-ba7e-47f6-80a6-ae01bcd89646 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:23.584222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af89a-ba7e-47f6-80a6-ae01bcd89646 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:23.591963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137af89a-ba7e-47f6-80a6-ae01bcd89646 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:23.593455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137af89a-ba7e-47f6-80a6-ae01bcd89646 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-137af89a-ba7e-47f6-80a6-ae01bcd89646 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:56:23.606522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:23.607530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:23.608443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:56:23.608443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:56:23.608443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:56:23.608443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:56:23.608443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:56:23.608443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:56:23.608443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:56:23.608443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:23.608443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 557/1096] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:56:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:23.613789 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9f85-af93-4f18-b530-6dbaa91b9010 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:23.626226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:23.724121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba9f85-af93-4f18-b530-6dbaa91b9010 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:23.760497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793f390-eecd-4d2c-b5ab-d3748546e8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:23.761111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f390-eecd-4d2c-b5ab-d3748546e8ae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:23.761389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f390-eecd-4d2c-b5ab-d3748546e8ae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:23.761663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793f390-eecd-4d2c-b5ab-d3748546e8ae None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:23.762740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793f390-eecd-4d2c-b5ab-d3748546e8ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:23.763046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793f390-eecd-4d2c-b5ab-d3748546e8ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:23.764499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793f390-eecd-4d2c-b5ab-d3748546e8ae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:23.776447 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 540/1097] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:56:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:23.782692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af89a-ba7e-47f6-80a6-ae01bcd89646 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:23.812903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:23.886788 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af89a-ba7e-47f6-80a6-ae01bcd89646 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:23.921469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f139e7-c9ab-40d2-a28e-a1821f7981bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:23.922096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f139e7-c9ab-40d2-a28e-a1821f7981bf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:23.922373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f139e7-c9ab-40d2-a28e-a1821f7981bf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:23.922640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f139e7-c9ab-40d2-a28e-a1821f7981bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:23.923348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f139e7-c9ab-40d2-a28e-a1821f7981bf None admin] RBAC: Adding query filter params (role.id=a850548ca8b0447bba66f8d63c407705, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=43a9ad5341b54c40a5427045660363c3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:23.923650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f139e7-c9ab-40d2-a28e-a1821f7981bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:23.925068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f139e7-c9ab-40d2-a28e-a1821f7981bf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:23.930711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 558/1098] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:56:23 2022] GET /identity/v3/role_assignments?user.id=43a9ad5341b54c40a5427045660363c3&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a850548ca8b0447bba66f8d63c407705 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:25.301569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfcb94-d39e-4df6-9163-e9a1f69ec8c5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:25.302141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfcb94-d39e-4df6-9163-e9a1f69ec8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:25.302226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdfcb94-d39e-4df6-9163-e9a1f69ec8c5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:25.303523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 541/1099] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:25.308662 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a139b9-55c7-475f-95a8-5da3ff6f0697 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:25.309395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a139b9-55c7-475f-95a8-5da3ff6f0697 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:25.309488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a139b9-55c7-475f-95a8-5da3ff6f0697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:25.328009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-34a139b9-55c7-475f-95a8-5da3ff6f0697 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:25.329208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-34a139b9-55c7-475f-95a8-5da3ff6f0697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:25.397367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 559/1100] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:25.673555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfcb94-d39e-4df6-9163-e9a1f69ec8c5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:25.742298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:25.777325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdfcb94-d39e-4df6-9163-e9a1f69ec8c5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:25.811523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac312c-52ec-447b-a7b4-e094f2c0cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:25.812136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac312c-52ec-447b-a7b4-e094f2c0cf45 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:25.812411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac312c-52ec-447b-a7b4-e094f2c0cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:25.812680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac312c-52ec-447b-a7b4-e094f2c0cf45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:25.822520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac312c-52ec-447b-a7b4-e094f2c0cf45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e7ac312c-52ec-447b-a7b4-e094f2c0cf45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:56:25.827241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:25.828363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:25.829550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 542/1101] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:56:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:25.834412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a139b9-55c7-475f-95a8-5da3ff6f0697 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:25.854753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:25.932155 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a139b9-55c7-475f-95a8-5da3ff6f0697 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:25.965266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff5e3d-1288-477a-af95-36addc349375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:25.965937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5e3d-1288-477a-af95-36addc349375 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:25.966218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5e3d-1288-477a-af95-36addc349375 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:25.966487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ff5e3d-1288-477a-af95-36addc349375 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:25.967144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff5e3d-1288-477a-af95-36addc349375 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:25.967444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff5e3d-1288-477a-af95-36addc349375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:25.968803 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ff5e3d-1288-477a-af95-36addc349375 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:25.976720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 560/1102] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:56:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:25.983186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac312c-52ec-447b-a7b4-e094f2c0cf45 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:26.090348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac312c-52ec-447b-a7b4-e094f2c0cf45 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:26.118852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:26.128142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c84d7-018f-47ae-b42e-2809d33d3dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:26.128772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c84d7-018f-47ae-b42e-2809d33d3dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:26.129049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c84d7-018f-47ae-b42e-2809d33d3dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:26.129322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4c84d7-018f-47ae-b42e-2809d33d3dd0 None admin] PATH_INFO: `/v3/users/70915623214646128afd2eb4dd871b92` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:26.130472 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c84d7-018f-47ae-b42e-2809d33d3dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=70915623214646128afd2eb4dd871b92)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:26.131911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4c84d7-018f-47ae-b42e-2809d33d3dd0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:26.133489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 543/1103] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:25 2022] GET /identity/v3/users/70915623214646128afd2eb4dd871b92 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:26.139906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff5e3d-1288-477a-af95-36addc349375 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:26.222673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:26.241021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ff5e3d-1288-477a-af95-36addc349375 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:26.273622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fe67b-9d8e-462b-8019-10283fee1736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:26.274252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe67b-9d8e-462b-8019-10283fee1736 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:26.274529 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe67b-9d8e-462b-8019-10283fee1736 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:26.274828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089fe67b-9d8e-462b-8019-10283fee1736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:26.341051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fe67b-9d8e-462b-8019-10283fee1736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:26.342526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089fe67b-9d8e-462b-8019-10283fee1736 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:26.421166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:26.440517 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 561/1104] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:56:26.446466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c84d7-018f-47ae-b42e-2809d33d3dd0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:26.546161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4c84d7-018f-47ae-b42e-2809d33d3dd0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:26.579448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a86019-241e-485f-bdc5-362a45fd67ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:26.580070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a86019-241e-485f-bdc5-362a45fd67ce None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:26.580350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a86019-241e-485f-bdc5-362a45fd67ce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:26.580623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a86019-241e-485f-bdc5-362a45fd67ce None admin] PATH_INFO: `/v3/projects/cbb28a11737b4187aef8af9c06619e8a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:26.581726 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a86019-241e-485f-bdc5-362a45fd67ce None admin] RBAC: Authorizing `identity:get_project(project_id=cbb28a11737b4187aef8af9c06619e8a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:26.583184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a86019-241e-485f-bdc5-362a45fd67ce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:26.584809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 544/1105] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:56:26 2022] GET /identity/v3/projects/cbb28a11737b4187aef8af9c06619e8a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:26.591511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fe67b-9d8e-462b-8019-10283fee1736 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:26.694887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089fe67b-9d8e-462b-8019-10283fee1736 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:26.733332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618319fa-4258-4d02-9490-290a0a6bf033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:26.733964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618319fa-4258-4d02-9490-290a0a6bf033 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:26.734239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618319fa-4258-4d02-9490-290a0a6bf033 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:26.734506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618319fa-4258-4d02-9490-290a0a6bf033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:26.735238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618319fa-4258-4d02-9490-290a0a6bf033 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=cbb28a11737b4187aef8af9c06619e8a, user.id=70915623214646128afd2eb4dd871b92) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:26.735542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618319fa-4258-4d02-9490-290a0a6bf033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:26.736988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618319fa-4258-4d02-9490-290a0a6bf033 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:26.743477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 562/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:56:26 2022] GET /identity/v3/role_assignments?user.id=70915623214646128afd2eb4dd871b92&scope.project.id=cbb28a11737b4187aef8af9c06619e8a&role.id=8b350845f331423cb0494e3449b58310 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:28.690129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf430f-bdd7-45f3-8237-7b03707c477b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:28.690690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf430f-bdd7-45f3-8237-7b03707c477b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:28.690690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf430f-bdd7-45f3-8237-7b03707c477b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:28.692003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 545/1107] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:28.697103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4d007-ed1a-47c5-9c2f-441de25d7a8c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:28.697485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4d007-ed1a-47c5-9c2f-441de25d7a8c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:28.697655 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f4d007-ed1a-47c5-9c2f-441de25d7a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:28.716089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f0f4d007-ed1a-47c5-9c2f-441de25d7a8c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:28.717283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f0f4d007-ed1a-47c5-9c2f-441de25d7a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:28.760054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:28.785824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 563/1108] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:29.054186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf430f-bdd7-45f3-8237-7b03707c477b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:29.119746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:29.159584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbf430f-bdd7-45f3-8237-7b03707c477b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:29.195336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2fada-3d68-4f5f-9cec-3839bd694305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:29.195987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2fada-3d68-4f5f-9cec-3839bd694305 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:29.196263 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2fada-3d68-4f5f-9cec-3839bd694305 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:29.196532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2fada-3d68-4f5f-9cec-3839bd694305 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:29.197722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2fada-3d68-4f5f-9cec-3839bd694305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:29.199232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2fada-3d68-4f5f-9cec-3839bd694305 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:29.201913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 546/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:29.207974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4d007-ed1a-47c5-9c2f-441de25d7a8c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:29.306279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f4d007-ed1a-47c5-9c2f-441de25d7a8c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:29.340583 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae7a48-f7ba-4b6d-adea-1c97638df550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:29.341281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae7a48-f7ba-4b6d-adea-1c97638df550 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:29.341481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae7a48-f7ba-4b6d-adea-1c97638df550 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:29.341747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae7a48-f7ba-4b6d-adea-1c97638df550 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:29.342404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae7a48-f7ba-4b6d-adea-1c97638df550 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:29.343831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae7a48-f7ba-4b6d-adea-1c97638df550 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:29.364462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 564/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:56:29 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:56:31.315748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef097a94-d395-4e78-8a29-90677ec54b2e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:31.316267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef097a94-d395-4e78-8a29-90677ec54b2e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:31.316267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef097a94-d395-4e78-8a29-90677ec54b2e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:31.317597 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 547/1111] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:31.322681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77292763-c33b-4718-ae38-d27f8aff67f8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:31.323076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77292763-c33b-4718-ae38-d27f8aff67f8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:31.323224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77292763-c33b-4718-ae38-d27f8aff67f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:31.341420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-77292763-c33b-4718-ae38-d27f8aff67f8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:31.342581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-77292763-c33b-4718-ae38-d27f8aff67f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:31.368446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:31.412168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 565/1112] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:31.685601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef097a94-d395-4e78-8a29-90677ec54b2e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:31.787393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef097a94-d395-4e78-8a29-90677ec54b2e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:31.818373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:31.823671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cc216-32db-4a0e-80bc-ea6526e41dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:31.824304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cc216-32db-4a0e-80bc-ea6526e41dab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:31.824707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cc216-32db-4a0e-80bc-ea6526e41dab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:31.824981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cc216-32db-4a0e-80bc-ea6526e41dab None admin] PATH_INFO: `/v3/roles/a631c57af3494b9c955799e0488518ae` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:31.826425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4cc216-32db-4a0e-80bc-ea6526e41dab None admin] RBAC: Authorizing `identity:get_role(role_id=a631c57af3494b9c955799e0488518ae)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:31.827799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4cc216-32db-4a0e-80bc-ea6526e41dab None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:31.829123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 548/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:31 2022] GET /identity/v3/roles/a631c57af3494b9c955799e0488518ae => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:31.834978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77292763-c33b-4718-ae38-d27f8aff67f8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:31.932309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77292763-c33b-4718-ae38-d27f8aff67f8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:31.966284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bed763-1df9-4713-a473-26a676306edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:31.966834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bed763-1df9-4713-a473-26a676306edb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:31.967116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bed763-1df9-4713-a473-26a676306edb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:31.967392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bed763-1df9-4713-a473-26a676306edb None admin] PATH_INFO: `/v3/users/ba30827d0d18452aa52754559f455b3a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:31.980272 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bed763-1df9-4713-a473-26a676306edb None admin] RBAC: Authorizing `identity:get_user(user_id=ba30827d0d18452aa52754559f455b3a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:31.982361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bed763-1df9-4713-a473-26a676306edb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:31.983976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 566/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:31 2022] GET /identity/v3/users/ba30827d0d18452aa52754559f455b3a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:31.989815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cc216-32db-4a0e-80bc-ea6526e41dab None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:32.088627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cc216-32db-4a0e-80bc-ea6526e41dab None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:32.123084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f8167-2366-4f1f-a928-015fab9af8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:32.123687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f8167-2366-4f1f-a928-015fab9af8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:32.123963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f8167-2366-4f1f-a928-015fab9af8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:32.124237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f8167-2366-4f1f-a928-015fab9af8a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:32.187963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:32.190357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f8167-2366-4f1f-a928-015fab9af8a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:32.192099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f8167-2366-4f1f-a928-015fab9af8a2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:32.289175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 549/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:56:32.295336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bed763-1df9-4713-a473-26a676306edb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:32.317659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:32.402780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bed763-1df9-4713-a473-26a676306edb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:32.436594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b8457-cb4f-4306-a846-85e4b61e02f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:32.437218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b8457-cb4f-4306-a846-85e4b61e02f6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:32.437495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b8457-cb4f-4306-a846-85e4b61e02f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:32.437791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b8457-cb4f-4306-a846-85e4b61e02f6 None admin] PATH_INFO: `/v3/projects/cbb28a11737b4187aef8af9c06619e8a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:32.438906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b8457-cb4f-4306-a846-85e4b61e02f6 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb28a11737b4187aef8af9c06619e8a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:32.440335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b8457-cb4f-4306-a846-85e4b61e02f6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:32.442014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 567/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:56:32 2022] GET /identity/v3/projects/cbb28a11737b4187aef8af9c06619e8a => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:32.448313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f8167-2366-4f1f-a928-015fab9af8a2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:32.544523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:32.547546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f8167-2366-4f1f-a928-015fab9af8a2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:32.583593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e84f7-0597-451c-a3b1-0b41f455435f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:32.584195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e84f7-0597-451c-a3b1-0b41f455435f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:32.584473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e84f7-0597-451c-a3b1-0b41f455435f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:32.584745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e84f7-0597-451c-a3b1-0b41f455435f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:32.585464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e84f7-0597-451c-a3b1-0b41f455435f None admin] RBAC: Adding query filter params (role.id=a631c57af3494b9c955799e0488518ae, scope.project.id=cbb28a11737b4187aef8af9c06619e8a, user.id=ba30827d0d18452aa52754559f455b3a) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:32.585796 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e84f7-0597-451c-a3b1-0b41f455435f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:32.587237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e84f7-0597-451c-a3b1-0b41f455435f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:32.593069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 550/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:56:32 2022] GET /identity/v3/role_assignments?user.id=ba30827d0d18452aa52754559f455b3a&scope.project.id=cbb28a11737b4187aef8af9c06619e8a&role.id=a631c57af3494b9c955799e0488518ae => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:34.513400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9094560-724a-4bb5-8da3-d4feaeb6dd67 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:34.513943 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9094560-724a-4bb5-8da3-d4feaeb6dd67 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:34.513943 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9094560-724a-4bb5-8da3-d4feaeb6dd67 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:34.515302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 568/1118] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:34.520791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344f976-b932-4270-9f59-6948f2b2f42a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:34.521167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344f976-b932-4270-9f59-6948f2b2f42a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:34.521338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344f976-b932-4270-9f59-6948f2b2f42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:34.539524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a344f976-b932-4270-9f59-6948f2b2f42a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:34.540724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a344f976-b932-4270-9f59-6948f2b2f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:34.607630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 551/1119] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:34.875210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9094560-724a-4bb5-8da3-d4feaeb6dd67 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:34.966081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:34.979211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9094560-724a-4bb5-8da3-d4feaeb6dd67 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:35.014983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b9788-9a78-4097-8da3-c6864aad1dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:35.015605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b9788-9a78-4097-8da3-c6864aad1dad None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:35.015877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b9788-9a78-4097-8da3-c6864aad1dad None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:35.016142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b9788-9a78-4097-8da3-c6864aad1dad None admin] PATH_INFO: `/v3/roles/a631c57af3494b9c955799e0488518ae` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:35.017578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b9788-9a78-4097-8da3-c6864aad1dad None admin] RBAC: Authorizing `identity:get_role(role_id=a631c57af3494b9c955799e0488518ae)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:35.018989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b9788-9a78-4097-8da3-c6864aad1dad None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:35.020383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 569/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:34 2022] GET /identity/v3/roles/a631c57af3494b9c955799e0488518ae => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:35.026525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344f976-b932-4270-9f59-6948f2b2f42a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:35.124674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:35.132129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344f976-b932-4270-9f59-6948f2b2f42a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:35.165962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100be611-f2ba-44c3-9253-4b51b4521401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:35.166591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be611-f2ba-44c3-9253-4b51b4521401 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:35.166871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be611-f2ba-44c3-9253-4b51b4521401 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:35.167142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100be611-f2ba-44c3-9253-4b51b4521401 None admin] PATH_INFO: `/v3/users/ba30827d0d18452aa52754559f455b3a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:35.168894 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100be611-f2ba-44c3-9253-4b51b4521401 None admin] RBAC: Authorizing `identity:get_user(user_id=ba30827d0d18452aa52754559f455b3a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:35.170361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100be611-f2ba-44c3-9253-4b51b4521401 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:35.171969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 552/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:35 2022] GET /identity/v3/users/ba30827d0d18452aa52754559f455b3a => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:35.177851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b9788-9a78-4097-8da3-c6864aad1dad None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:35.277169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b9788-9a78-4097-8da3-c6864aad1dad None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:35.310568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a46c85-6471-4ed5-b684-126d36ae6e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:35.311195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a46c85-6471-4ed5-b684-126d36ae6e38 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:35.311478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a46c85-6471-4ed5-b684-126d36ae6e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:35.311749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a46c85-6471-4ed5-b684-126d36ae6e38 None admin] PATH_INFO: `/v3/projects/cbb28a11737b4187aef8af9c06619e8a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:35.312888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a46c85-6471-4ed5-b684-126d36ae6e38 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb28a11737b4187aef8af9c06619e8a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:35.314356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a46c85-6471-4ed5-b684-126d36ae6e38 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:35.316003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 570/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:56:35 2022] GET /identity/v3/projects/cbb28a11737b4187aef8af9c06619e8a => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:35.322041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100be611-f2ba-44c3-9253-4b51b4521401 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:35.421554 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100be611-f2ba-44c3-9253-4b51b4521401 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:35.454403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607e8b4-b4e8-4fb5-905b-4393f302d592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:35.455057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607e8b4-b4e8-4fb5-905b-4393f302d592 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:35.455337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607e8b4-b4e8-4fb5-905b-4393f302d592 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:35.455611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607e8b4-b4e8-4fb5-905b-4393f302d592 None admin] PATH_INFO: `/v3/projects/cbb28a11737b4187aef8af9c06619e8a/users/ba30827d0d18452aa52754559f455b3a/roles/a631c57af3494b9c955799e0488518ae` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:35.457739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607e8b4-b4e8-4fb5-905b-4393f302d592 None admin] RBAC: Authorizing `identity:create_grant(project_id=cbb28a11737b4187aef8af9c06619e8a, user_id=ba30827d0d18452aa52754559f455b3a, role_id=a631c57af3494b9c955799e0488518ae)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:35.459193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607e8b4-b4e8-4fb5-905b-4393f302d592 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:35.472939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 553/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:56:35 2022] PUT /identity/v3/projects/cbb28a11737b4187aef8af9c06619e8a/users/ba30827d0d18452aa52754559f455b3a/roles/a631c57af3494b9c955799e0488518ae => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:56:37.416492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d94df-e958-40b1-a784-36a4ecc9c89d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:37.417054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d94df-e958-40b1-a784-36a4ecc9c89d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:37.417054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d94df-e958-40b1-a784-36a4ecc9c89d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:37.418465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 571/1124] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:37.423957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f02a13-211b-4d37-9357-43b950a6b88a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:37.424367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f02a13-211b-4d37-9357-43b950a6b88a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:37.424515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f02a13-211b-4d37-9357-43b950a6b88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:37.443275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b7f02a13-211b-4d37-9357-43b950a6b88a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:37.444527 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b7f02a13-211b-4d37-9357-43b950a6b88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:37.463786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:37.519912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 554/1125] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:37.789426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d94df-e958-40b1-a784-36a4ecc9c89d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:37.815257 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:37.892013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d94df-e958-40b1-a784-36a4ecc9c89d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:37.926083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f612dea-bfe3-421c-8e4b-be5ae3ee0b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:37.926720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f612dea-bfe3-421c-8e4b-be5ae3ee0b4b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:37.927002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f612dea-bfe3-421c-8e4b-be5ae3ee0b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:37.927279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f612dea-bfe3-421c-8e4b-be5ae3ee0b4b None admin] PATH_INFO: `/v3/roles/a631c57af3494b9c955799e0488518ae` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:37.928687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f612dea-bfe3-421c-8e4b-be5ae3ee0b4b None admin] RBAC: Authorizing `identity:get_role(role_id=a631c57af3494b9c955799e0488518ae)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:37.930103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f612dea-bfe3-421c-8e4b-be5ae3ee0b4b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:37.931443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 572/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:37 2022] GET /identity/v3/roles/a631c57af3494b9c955799e0488518ae => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:37.938545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f02a13-211b-4d37-9357-43b950a6b88a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:38.036508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f02a13-211b-4d37-9357-43b950a6b88a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:38.071209 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c8d5c-dac6-4cfb-8e5c-32893945b5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:38.071754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c8d5c-dac6-4cfb-8e5c-32893945b5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:38.072026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c8d5c-dac6-4cfb-8e5c-32893945b5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:38.072293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c8d5c-dac6-4cfb-8e5c-32893945b5e7 None admin] PATH_INFO: `/v3/users/ba30827d0d18452aa52754559f455b3a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:38.073501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c8d5c-dac6-4cfb-8e5c-32893945b5e7 None admin] RBAC: Authorizing `identity:get_user(user_id=ba30827d0d18452aa52754559f455b3a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:38.075478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45c8d5c-dac6-4cfb-8e5c-32893945b5e7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:38.077115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 555/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:37 2022] GET /identity/v3/users/ba30827d0d18452aa52754559f455b3a => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:38.083146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f612dea-bfe3-421c-8e4b-be5ae3ee0b4b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:38.175886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:38.185079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f612dea-bfe3-421c-8e4b-be5ae3ee0b4b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:38.218792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c5d1c-4c3f-43d4-92ac-f81c14b3fe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:38.219395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c5d1c-4c3f-43d4-92ac-f81c14b3fe9e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:38.219674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c5d1c-4c3f-43d4-92ac-f81c14b3fe9e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:38.219947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c5d1c-4c3f-43d4-92ac-f81c14b3fe9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:38.287791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c5d1c-4c3f-43d4-92ac-f81c14b3fe9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:38.289250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c5d1c-4c3f-43d4-92ac-f81c14b3fe9e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:38.383042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:38.394090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 573/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:56:38.400058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c8d5c-dac6-4cfb-8e5c-32893945b5e7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:38.422144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:38.502073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c8d5c-dac6-4cfb-8e5c-32893945b5e7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:38.535557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8cf7e-2824-4818-99a1-a2f52123bbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:38.536183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8cf7e-2824-4818-99a1-a2f52123bbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:38.536463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8cf7e-2824-4818-99a1-a2f52123bbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:38.536739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8cf7e-2824-4818-99a1-a2f52123bbc3 None admin] PATH_INFO: `/v3/projects/cbb28a11737b4187aef8af9c06619e8a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:38.537876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8cf7e-2824-4818-99a1-a2f52123bbc3 None admin] RBAC: Authorizing `identity:get_project(project_id=cbb28a11737b4187aef8af9c06619e8a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:38.539310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8cf7e-2824-4818-99a1-a2f52123bbc3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:38.540941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 556/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:56:38 2022] GET /identity/v3/projects/cbb28a11737b4187aef8af9c06619e8a => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:38.547749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c5d1c-4c3f-43d4-92ac-f81c14b3fe9e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:38.648577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c5d1c-4c3f-43d4-92ac-f81c14b3fe9e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:38.683198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15916c98-c48a-41b0-9cd1-8dc1aff97f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:38.684381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15916c98-c48a-41b0-9cd1-8dc1aff97f1a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:38.684653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15916c98-c48a-41b0-9cd1-8dc1aff97f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:38.684921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15916c98-c48a-41b0-9cd1-8dc1aff97f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:38.685633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15916c98-c48a-41b0-9cd1-8dc1aff97f1a None admin] RBAC: Adding query filter params (role.id=a631c57af3494b9c955799e0488518ae, scope.project.id=cbb28a11737b4187aef8af9c06619e8a, user.id=ba30827d0d18452aa52754559f455b3a) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:38.685966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15916c98-c48a-41b0-9cd1-8dc1aff97f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:38.687467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15916c98-c48a-41b0-9cd1-8dc1aff97f1a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:38.693063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 574/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:56:38 2022] GET /identity/v3/role_assignments?user.id=ba30827d0d18452aa52754559f455b3a&scope.project.id=cbb28a11737b4187aef8af9c06619e8a&role.id=a631c57af3494b9c955799e0488518ae => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:40.656939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ae14d-7f85-406c-9968-21248897044e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:40.657497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ae14d-7f85-406c-9968-21248897044e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:40.657497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9ae14d-7f85-406c-9968-21248897044e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:40.658856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 557/1131] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:40.664270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d589b-b985-4d53-bc32-9f6b46adf6f8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:40.664703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d589b-b985-4d53-bc32-9f6b46adf6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:40.664822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d589b-b985-4d53-bc32-9f6b46adf6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:40.685376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-587d589b-b985-4d53-bc32-9f6b46adf6f8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:40.686602 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-587d589b-b985-4d53-bc32-9f6b46adf6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:40.736528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:40.754133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 575/1132] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:41.034868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ae14d-7f85-406c-9968-21248897044e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:41.128525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:41.142660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9ae14d-7f85-406c-9968-21248897044e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:41.177670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024a4a6-77ca-4ea9-9f44-e210ded03f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:41.178325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024a4a6-77ca-4ea9-9f44-e210ded03f79 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:41.178599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024a4a6-77ca-4ea9-9f44-e210ded03f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:41.178870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024a4a6-77ca-4ea9-9f44-e210ded03f79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:41.180035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024a4a6-77ca-4ea9-9f44-e210ded03f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:41.181519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024a4a6-77ca-4ea9-9f44-e210ded03f79 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:41.183162 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 558/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:41.189214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d589b-b985-4d53-bc32-9f6b46adf6f8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:41.290177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d589b-b985-4d53-bc32-9f6b46adf6f8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:41.323838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a11e3-fcf7-408a-b155-3a933079f2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:41.325018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a11e3-fcf7-408a-b155-3a933079f2e4 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:41.325294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a11e3-fcf7-408a-b155-3a933079f2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:41.325565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a11e3-fcf7-408a-b155-3a933079f2e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:41.326235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a11e3-fcf7-408a-b155-3a933079f2e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:41.327667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a11e3-fcf7-408a-b155-3a933079f2e4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:41.353811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 576/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:56:41 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:56:43.311841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3d7c-6e86-4ccc-a5bf-3e049a8e01cf None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:43.312393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3d7c-6e86-4ccc-a5bf-3e049a8e01cf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:43.312393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3d7c-6e86-4ccc-a5bf-3e049a8e01cf None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:43.313945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 559/1135] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:43.319044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696acac-cc4c-45f8-ba59-061a96949532 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:43.319448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696acac-cc4c-45f8-ba59-061a96949532 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:43.319590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c696acac-cc4c-45f8-ba59-061a96949532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:43.337842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c696acac-cc4c-45f8-ba59-061a96949532 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:43.338965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c696acac-cc4c-45f8-ba59-061a96949532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:43.368192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:43.407904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 577/1136] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:56:43.681953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3d7c-6e86-4ccc-a5bf-3e049a8e01cf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:43.783810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3d7c-6e86-4ccc-a5bf-3e049a8e01cf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:43.820598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3189a4-f43f-4be3-9d8a-6f17c11b1c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:43.821244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3189a4-f43f-4be3-9d8a-6f17c11b1c6f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:43.821516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3189a4-f43f-4be3-9d8a-6f17c11b1c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:43.821813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3189a4-f43f-4be3-9d8a-6f17c11b1c6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:43.823002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3189a4-f43f-4be3-9d8a-6f17c11b1c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:43.825112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3189a4-f43f-4be3-9d8a-6f17c11b1c6f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:43.826810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 560/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:56:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:43.832770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696acac-cc4c-45f8-ba59-061a96949532 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:43.929867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c696acac-cc4c-45f8-ba59-061a96949532 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:43.964115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22181bb-ce91-4161-b729-635d7b51c3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:43.964664 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22181bb-ce91-4161-b729-635d7b51c3d5 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:43.965551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22181bb-ce91-4161-b729-635d7b51c3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:43.965927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22181bb-ce91-4161-b729-635d7b51c3d5 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:43.966569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22181bb-ce91-4161-b729-635d7b51c3d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:43.968001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22181bb-ce91-4161-b729-635d7b51c3d5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:43.992870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 578/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:56:43 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:56:45.946593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef14e1-73e8-46a8-b2d3-2e172e0e271c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:45.947153 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef14e1-73e8-46a8-b2d3-2e172e0e271c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:45.947153 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef14e1-73e8-46a8-b2d3-2e172e0e271c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:45.948457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 561/1139] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:45.953797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05294555-e2f6-455c-acf4-ef5f6e4708d9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:45.954264 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05294555-e2f6-455c-acf4-ef5f6e4708d9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:45.954375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05294555-e2f6-455c-acf4-ef5f6e4708d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:45.972819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-05294555-e2f6-455c-acf4-ef5f6e4708d9 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:45.974078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-05294555-e2f6-455c-acf4-ef5f6e4708d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:45.993634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:46.050504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 579/1140] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:46.319002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef14e1-73e8-46a8-b2d3-2e172e0e271c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:46.347281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:46.430285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef14e1-73e8-46a8-b2d3-2e172e0e271c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:46.463994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a259b-899c-4122-b290-436ed138d462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:46.465154 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a259b-899c-4122-b290-436ed138d462 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:46.465435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a259b-899c-4122-b290-436ed138d462 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:46.465705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a259b-899c-4122-b290-436ed138d462 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:46.475961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a259b-899c-4122-b290-436ed138d462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5b6a259b-899c-4122-b290-436ed138d462 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:56:46.481384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:46.482623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:46.483886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 562/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:56:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:46.488708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05294555-e2f6-455c-acf4-ef5f6e4708d9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:46.590902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05294555-e2f6-455c-acf4-ef5f6e4708d9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:46.627508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6fe5f-b8f0-4a8f-adba-d5ce99960e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:46.628118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6fe5f-b8f0-4a8f-adba-d5ce99960e32 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:46.628394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6fe5f-b8f0-4a8f-adba-d5ce99960e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:46.628663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6fe5f-b8f0-4a8f-adba-d5ce99960e32 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:46.629333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6fe5f-b8f0-4a8f-adba-d5ce99960e32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:46.629632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6fe5f-b8f0-4a8f-adba-d5ce99960e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:46.631034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6fe5f-b8f0-4a8f-adba-d5ce99960e32 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:46.640215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 580/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:56:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:46.646252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a259b-899c-4122-b290-436ed138d462 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:46.739586 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:46.757224 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a259b-899c-4122-b290-436ed138d462 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:46.792061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804e25b-631e-4a2f-afce-783a3d259582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:46.792692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804e25b-631e-4a2f-afce-783a3d259582 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:46.793653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804e25b-631e-4a2f-afce-783a3d259582 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:46.794189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804e25b-631e-4a2f-afce-783a3d259582 None admin] PATH_INFO: `/v3/users/31592d16f2724c6b805da55e1b900d18` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:46.808065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804e25b-631e-4a2f-afce-783a3d259582 None admin] RBAC: Authorizing `identity:get_user(user_id=31592d16f2724c6b805da55e1b900d18)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:46.811590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804e25b-631e-4a2f-afce-783a3d259582 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:46.812012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 563/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:46 2022] GET /identity/v3/users/31592d16f2724c6b805da55e1b900d18 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:46.818047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6fe5f-b8f0-4a8f-adba-d5ce99960e32 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:46.836784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:46.924746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6fe5f-b8f0-4a8f-adba-d5ce99960e32 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:46.959001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b568a-5481-4075-b2b2-fe7a52f98d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:46.959678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b568a-5481-4075-b2b2-fe7a52f98d2c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:46.959879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b568a-5481-4075-b2b2-fe7a52f98d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:46.960152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b568a-5481-4075-b2b2-fe7a52f98d2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:47.037215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b568a-5481-4075-b2b2-fe7a52f98d2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:47.038751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b568a-5481-4075-b2b2-fe7a52f98d2c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:47.057010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:47.146359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 581/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:56:47.152991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804e25b-631e-4a2f-afce-783a3d259582 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:47.255851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804e25b-631e-4a2f-afce-783a3d259582 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:47.290200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad0cc-d5ea-4cf1-915c-8db81b116141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:47.290828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad0cc-d5ea-4cf1-915c-8db81b116141 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:47.291103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad0cc-d5ea-4cf1-915c-8db81b116141 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:47.291378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad0cc-d5ea-4cf1-915c-8db81b116141 None admin] PATH_INFO: `/v3/projects/c57383045fd74273aa0300b2c03e9e17` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:47.292495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ad0cc-d5ea-4cf1-915c-8db81b116141 None admin] RBAC: Authorizing `identity:get_project(project_id=c57383045fd74273aa0300b2c03e9e17)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:47.294551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ad0cc-d5ea-4cf1-915c-8db81b116141 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:47.296396 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 564/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:56:47 2022] GET /identity/v3/projects/c57383045fd74273aa0300b2c03e9e17 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:47.302672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b568a-5481-4075-b2b2-fe7a52f98d2c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:47.407241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b568a-5481-4075-b2b2-fe7a52f98d2c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:47.423995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:47.443076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb4ad9-9e25-4963-8ae5-1056230c758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:47.444233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb4ad9-9e25-4963-8ae5-1056230c758e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:47.444517 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb4ad9-9e25-4963-8ae5-1056230c758e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:47.444786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb4ad9-9e25-4963-8ae5-1056230c758e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:47.445508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb4ad9-9e25-4963-8ae5-1056230c758e None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=c57383045fd74273aa0300b2c03e9e17, user.id=31592d16f2724c6b805da55e1b900d18) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:47.445838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb4ad9-9e25-4963-8ae5-1056230c758e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:47.447260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb4ad9-9e25-4963-8ae5-1056230c758e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:47.454886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 582/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:56:47 2022] GET /identity/v3/role_assignments?user.id=31592d16f2724c6b805da55e1b900d18&scope.project.id=c57383045fd74273aa0300b2c03e9e17&role.id=8b350845f331423cb0494e3449b58310 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:49.360638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6adc68-a7b3-4501-a4a7-f0db9fbae8cf None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:49.361149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6adc68-a7b3-4501-a4a7-f0db9fbae8cf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:49.361149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6adc68-a7b3-4501-a4a7-f0db9fbae8cf None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:49.362494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 565/1147] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:49.368584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04786558-0666-4e83-9050-172fb9f000ed None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:49.368993 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04786558-0666-4e83-9050-172fb9f000ed None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:49.369128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04786558-0666-4e83-9050-172fb9f000ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:49.388060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-04786558-0666-4e83-9050-172fb9f000ed None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:49.389270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04786558-0666-4e83-9050-172fb9f000ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:49.456981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 583/1148] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:49.728250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6adc68-a7b3-4501-a4a7-f0db9fbae8cf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:49.740684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:49.838240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6adc68-a7b3-4501-a4a7-f0db9fbae8cf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:49.874825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e3370-de91-4786-aabb-b1c1fc95d48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:49.875459 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e3370-de91-4786-aabb-b1c1fc95d48b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:49.875737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e3370-de91-4786-aabb-b1c1fc95d48b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:49.876008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e3370-de91-4786-aabb-b1c1fc95d48b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:49.888090 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e3370-de91-4786-aabb-b1c1fc95d48b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-033e3370-de91-4786-aabb-b1c1fc95d48b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:56:49.893075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:49.894269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:49.895255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 566/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:56:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:49.900637 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04786558-0666-4e83-9050-172fb9f000ed None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:50.001301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04786558-0666-4e83-9050-172fb9f000ed None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:50.012362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:50.036912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7e344-0afe-4a8f-ab62-91bc7444c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:50.037522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7e344-0afe-4a8f-ab62-91bc7444c879 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:50.037823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7e344-0afe-4a8f-ab62-91bc7444c879 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:50.038099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7e344-0afe-4a8f-ab62-91bc7444c879 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:50.038760 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7e344-0afe-4a8f-ab62-91bc7444c879 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:50.039062 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7e344-0afe-4a8f-ab62-91bc7444c879 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:50.040436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7e344-0afe-4a8f-ab62-91bc7444c879 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:50.048952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 584/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:56:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:50.055109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e3370-de91-4786-aabb-b1c1fc95d48b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:50.125563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:50.157537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e3370-de91-4786-aabb-b1c1fc95d48b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:50.191509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee50ce-25ce-4488-a37f-6829792aa0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:50.192136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee50ce-25ce-4488-a37f-6829792aa0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:50.192412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee50ce-25ce-4488-a37f-6829792aa0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:50.192686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee50ce-25ce-4488-a37f-6829792aa0c3 None admin] PATH_INFO: `/v3/users/31592d16f2724c6b805da55e1b900d18` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:50.193821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee50ce-25ce-4488-a37f-6829792aa0c3 None admin] RBAC: Authorizing `identity:get_user(user_id=31592d16f2724c6b805da55e1b900d18)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:50.195260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee50ce-25ce-4488-a37f-6829792aa0c3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:50.196832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 567/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:50 2022] GET /identity/v3/users/31592d16f2724c6b805da55e1b900d18 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:50.203632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7e344-0afe-4a8f-ab62-91bc7444c879 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:50.303874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7e344-0afe-4a8f-ab62-91bc7444c879 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:50.336001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf7b79-2bfb-49d3-96f3-ca2264af9849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:50.336623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf7b79-2bfb-49d3-96f3-ca2264af9849 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:50.336906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf7b79-2bfb-49d3-96f3-ca2264af9849 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:50.337181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf7b79-2bfb-49d3-96f3-ca2264af9849 None admin] PATH_INFO: `/v3/projects/c57383045fd74273aa0300b2c03e9e17` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:50.338313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf7b79-2bfb-49d3-96f3-ca2264af9849 None admin] RBAC: Authorizing `identity:get_project(project_id=c57383045fd74273aa0300b2c03e9e17)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:50.340293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf7b79-2bfb-49d3-96f3-ca2264af9849 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:50.341927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 585/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:56:50 2022] GET /identity/v3/projects/c57383045fd74273aa0300b2c03e9e17 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:50.348054 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee50ce-25ce-4488-a37f-6829792aa0c3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:50.445342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee50ce-25ce-4488-a37f-6829792aa0c3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:50.478855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c801e0b-10b1-452d-860c-0f5ed633bcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:50.479545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c801e0b-10b1-452d-860c-0f5ed633bcba None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:50.479824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c801e0b-10b1-452d-860c-0f5ed633bcba None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:50.480102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c801e0b-10b1-452d-860c-0f5ed633bcba None admin] PATH_INFO: `/v3/projects/c57383045fd74273aa0300b2c03e9e17/users/31592d16f2724c6b805da55e1b900d18/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:50.482809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c801e0b-10b1-452d-860c-0f5ed633bcba None admin] RBAC: Authorizing `identity:create_grant(project_id=c57383045fd74273aa0300b2c03e9e17, user_id=31592d16f2724c6b805da55e1b900d18, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:50.484284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c801e0b-10b1-452d-860c-0f5ed633bcba None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:50.494225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:50.501888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 568/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:56:50 2022] PUT /identity/v3/projects/c57383045fd74273aa0300b2c03e9e17/users/31592d16f2724c6b805da55e1b900d18/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:56:52.416614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606bc0d-4e23-4a25-a331-5d58e1586a6e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:52.417132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606bc0d-4e23-4a25-a331-5d58e1586a6e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:52.417132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606bc0d-4e23-4a25-a331-5d58e1586a6e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:52.419179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 586/1154] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:52.425149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2626908-6f4c-4f0e-9f44-efdb727c154e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:52.425707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2626908-6f4c-4f0e-9f44-efdb727c154e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:52.425707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2626908-6f4c-4f0e-9f44-efdb727c154e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:52.444729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b2626908-6f4c-4f0e-9f44-efdb727c154e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:52.446006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b2626908-6f4c-4f0e-9f44-efdb727c154e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:52.518112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 569/1155] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:52.808119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606bc0d-4e23-4a25-a331-5d58e1586a6e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:52.852698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:52.913092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606bc0d-4e23-4a25-a331-5d58e1586a6e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:52.951623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916fdc9-fd8b-4560-abaa-4043c839b23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:52.952271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916fdc9-fd8b-4560-abaa-4043c839b23c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:52.952547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916fdc9-fd8b-4560-abaa-4043c839b23c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:52.952815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916fdc9-fd8b-4560-abaa-4043c839b23c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:52.965545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916fdc9-fd8b-4560-abaa-4043c839b23c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5916fdc9-fd8b-4560-abaa-4043c839b23c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:56:52.970509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:52.971726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:56:52.972723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 587/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:56:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:52.978124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2626908-6f4c-4f0e-9f44-efdb727c154e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:53.088442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2626908-6f4c-4f0e-9f44-efdb727c154e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:53.116612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:53.125756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41100c68-2060-45e6-b35d-6e25445ba7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:53.126392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41100c68-2060-45e6-b35d-6e25445ba7c0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:53.126668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41100c68-2060-45e6-b35d-6e25445ba7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:53.126940 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41100c68-2060-45e6-b35d-6e25445ba7c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:53.127614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41100c68-2060-45e6-b35d-6e25445ba7c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:53.127911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41100c68-2060-45e6-b35d-6e25445ba7c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:53.129295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41100c68-2060-45e6-b35d-6e25445ba7c0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:53.138115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 570/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:56:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:53.144101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916fdc9-fd8b-4560-abaa-4043c839b23c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:53.247607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:53.250709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916fdc9-fd8b-4560-abaa-4043c839b23c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:53.286168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda988b7-b02e-4947-a44c-a050e871fbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:53.286805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda988b7-b02e-4947-a44c-a050e871fbd3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:53.287079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda988b7-b02e-4947-a44c-a050e871fbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:53.287347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda988b7-b02e-4947-a44c-a050e871fbd3 None admin] PATH_INFO: `/v3/users/31592d16f2724c6b805da55e1b900d18` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:53.288505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda988b7-b02e-4947-a44c-a050e871fbd3 None admin] RBAC: Authorizing `identity:get_user(user_id=31592d16f2724c6b805da55e1b900d18)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:53.290016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda988b7-b02e-4947-a44c-a050e871fbd3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:53.291734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 588/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:56:53 2022] GET /identity/v3/users/31592d16f2724c6b805da55e1b900d18 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:53.298319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41100c68-2060-45e6-b35d-6e25445ba7c0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:53.413594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41100c68-2060-45e6-b35d-6e25445ba7c0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:53.447332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a18f-0a54-4d11-98ad-89fb7b7d5c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:53.448015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a18f-0a54-4d11-98ad-89fb7b7d5c0d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:53.448200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a18f-0a54-4d11-98ad-89fb7b7d5c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:53.448473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962a18f-0a54-4d11-98ad-89fb7b7d5c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:53.507753 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:53.519319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962a18f-0a54-4d11-98ad-89fb7b7d5c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:53.520777 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962a18f-0a54-4d11-98ad-89fb7b7d5c0d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:53.626516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 571/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:56:53.632846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda988b7-b02e-4947-a44c-a050e871fbd3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:53.734177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda988b7-b02e-4947-a44c-a050e871fbd3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:53.769269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f581da3-f636-4093-a22a-b9eb8e4e278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:53.769921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f581da3-f636-4093-a22a-b9eb8e4e278c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:53.770203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f581da3-f636-4093-a22a-b9eb8e4e278c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:53.770476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f581da3-f636-4093-a22a-b9eb8e4e278c None admin] PATH_INFO: `/v3/projects/c57383045fd74273aa0300b2c03e9e17` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:53.771600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f581da3-f636-4093-a22a-b9eb8e4e278c None admin] RBAC: Authorizing `identity:get_project(project_id=c57383045fd74273aa0300b2c03e9e17)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:53.773034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f581da3-f636-4093-a22a-b9eb8e4e278c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:53.774726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 589/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:56:53 2022] GET /identity/v3/projects/c57383045fd74273aa0300b2c03e9e17 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:53.780968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a18f-0a54-4d11-98ad-89fb7b7d5c0d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:53.877032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:53.887483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962a18f-0a54-4d11-98ad-89fb7b7d5c0d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:53.922730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b03c3-93c7-41a5-9b5d-fa91282889ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:53.923331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b03c3-93c7-41a5-9b5d-fa91282889ed None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:53.923605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b03c3-93c7-41a5-9b5d-fa91282889ed None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:53.923873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b03c3-93c7-41a5-9b5d-fa91282889ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:53.924596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b03c3-93c7-41a5-9b5d-fa91282889ed None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=c57383045fd74273aa0300b2c03e9e17, user.id=31592d16f2724c6b805da55e1b900d18) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:53.924902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b03c3-93c7-41a5-9b5d-fa91282889ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:53.926362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b03c3-93c7-41a5-9b5d-fa91282889ed None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:53.932231 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 572/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:56:53 2022] GET /identity/v3/role_assignments?user.id=31592d16f2724c6b805da55e1b900d18&scope.project.id=c57383045fd74273aa0300b2c03e9e17&role.id=8b350845f331423cb0494e3449b58310 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:56:56.079676 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845c213-66dd-4419-a7a3-b531978b1bb1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:56.080198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845c213-66dd-4419-a7a3-b531978b1bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:56.080307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845c213-66dd-4419-a7a3-b531978b1bb1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:56.081567 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 590/1162] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:56.086896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5708d-61b7-4b88-8f71-2e858d4c838c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:56.087307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5708d-61b7-4b88-8f71-2e858d4c838c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:56.087456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5708d-61b7-4b88-8f71-2e858d4c838c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:56.107009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-87f5708d-61b7-4b88-8f71-2e858d4c838c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:56.108218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-87f5708d-61b7-4b88-8f71-2e858d4c838c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:56.174374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 573/1163] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:56.442859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845c213-66dd-4419-a7a3-b531978b1bb1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:56.485313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:56.551282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845c213-66dd-4419-a7a3-b531978b1bb1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:56.584687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a29ddc-56b3-4c3c-ad50-8e153ddb3c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:56.585404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a29ddc-56b3-4c3c-ad50-8e153ddb3c23 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:56.585686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a29ddc-56b3-4c3c-ad50-8e153ddb3c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:56.585991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a29ddc-56b3-4c3c-ad50-8e153ddb3c23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:56.654602 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a29ddc-56b3-4c3c-ad50-8e153ddb3c23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:56.656054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a29ddc-56b3-4c3c-ad50-8e153ddb3c23 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:56.688105 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:56.754134 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 591/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:56:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:56:56.760055 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5708d-61b7-4b88-8f71-2e858d4c838c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:56.845624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:56.862226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5708d-61b7-4b88-8f71-2e858d4c838c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:56.895677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a75bb0-7b0b-416a-8bc6-1b7d1589bd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:56.896311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a75bb0-7b0b-416a-8bc6-1b7d1589bd02 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:56.896591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a75bb0-7b0b-416a-8bc6-1b7d1589bd02 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:56.896869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a75bb0-7b0b-416a-8bc6-1b7d1589bd02 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:56.904222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a75bb0-7b0b-416a-8bc6-1b7d1589bd02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:56.905664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a75bb0-7b0b-416a-8bc6-1b7d1589bd02 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d4a75bb0-7b0b-416a-8bc6-1b7d1589bd02 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:56:56.916904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:56:56.919150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:56:56.920187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 574/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Aug 19 20:56:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:56:56.924662 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a29ddc-56b3-4c3c-ad50-8e153ddb3c23 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:57.022423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a29ddc-56b3-4c3c-ad50-8e153ddb3c23 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:57.057191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df55741-f736-49eb-b6f3-cc047c953a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:57.057821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df55741-f736-49eb-b6f3-cc047c953a7b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:57.058205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df55741-f736-49eb-b6f3-cc047c953a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:57.058585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df55741-f736-49eb-b6f3-cc047c953a7b None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:57.059234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df55741-f736-49eb-b6f3-cc047c953a7b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:56:57.059538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df55741-f736-49eb-b6f3-cc047c953a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:57.060902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df55741-f736-49eb-b6f3-cc047c953a7b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:57.063244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:56:57.068717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 592/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Fri Aug 19 20:56:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:56:58.970878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d88353-7c49-4c5d-a408-594e0e8e3290 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:58.971431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d88353-7c49-4c5d-a408-594e0e8e3290 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:58.971431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d88353-7c49-4c5d-a408-594e0e8e3290 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:58.972780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 575/1167] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:56:58.977879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6471-861f-48cd-b766-2ca5cf527af4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:58.978268 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6471-861f-48cd-b766-2ca5cf527af4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:58.978426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b6471-861f-48cd-b766-2ca5cf527af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:58.997475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-914b6471-861f-48cd-b766-2ca5cf527af4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:56:58.998690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-914b6471-861f-48cd-b766-2ca5cf527af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:56:59.064413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 593/1168] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:56:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:56:59.332986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d88353-7c49-4c5d-a408-594e0e8e3290 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:56:59.433614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d88353-7c49-4c5d-a408-594e0e8e3290 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:56:59.466634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0e56eb-77dd-48bc-833a-9e70c0cdbdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:56:59.467239 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e56eb-77dd-48bc-833a-9e70c0cdbdf8 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:56:59.467519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e56eb-77dd-48bc-833a-9e70c0cdbdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:56:59.467792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e56eb-77dd-48bc-833a-9e70c0cdbdf8 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:56:59.468448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e56eb-77dd-48bc-833a-9e70c0cdbdf8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:56:59.469834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0e56eb-77dd-48bc-833a-9e70c0cdbdf8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:56:59.486149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 576/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Fri Aug 19 20:56:59 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:57:01.450824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a591c5-677e-483c-b71a-e77204dfe46b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:01.451437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a591c5-677e-483c-b71a-e77204dfe46b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:01.451437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a591c5-677e-483c-b71a-e77204dfe46b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:01.452869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 594/1170] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:57:01.458744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78240-0787-4103-9695-512b8275758c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:01.459501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78240-0787-4103-9695-512b8275758c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:01.459690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f78240-0787-4103-9695-512b8275758c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:01.467913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:01.480647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-75f78240-0787-4103-9695-512b8275758c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:57:01.482085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-75f78240-0787-4103-9695-512b8275758c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:57:01.554328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 577/1171] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:57:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:57:01.836947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a591c5-677e-483c-b71a-e77204dfe46b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:01.948516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a591c5-677e-483c-b71a-e77204dfe46b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:01.979174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:01.986590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b0a18e-5e37-4249-815b-ab0217cccb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:01.987339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0a18e-5e37-4249-815b-ab0217cccb67 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:01.987744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0a18e-5e37-4249-815b-ab0217cccb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:01.988177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b0a18e-5e37-4249-815b-ab0217cccb67 None admin] PATH_INFO: `/v3/domains/9cdf061bf3cf4acfb483512e4fea73a0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:02.004387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0a18e-5e37-4249-815b-ab0217cccb67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9cdf061bf3cf4acfb483512e4fea73a0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:02.005893 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b0a18e-5e37-4249-815b-ab0217cccb67 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:02.007637 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 595/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Fri Aug 19 20:57:01 2022] GET /identity/v3/domains/9cdf061bf3cf4acfb483512e4fea73a0 => generated 267 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:57:02.014945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f78240-0787-4103-9695-512b8275758c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:02.188667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f78240-0787-4103-9695-512b8275758c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:02.224843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4023d71d-ec1e-4bc0-8656-035280d09ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:02.225461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023d71d-ec1e-4bc0-8656-035280d09ac8 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:02.225736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023d71d-ec1e-4bc0-8656-035280d09ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:02.226035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4023d71d-ec1e-4bc0-8656-035280d09ac8 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:02.226687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023d71d-ec1e-4bc0-8656-035280d09ac8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:02.228119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4023d71d-ec1e-4bc0-8656-035280d09ac8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:02.234300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:02.262273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 578/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:57:02 2022] POST /identity/v3/projects => generated 361 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:57:04.355892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a060b-58d5-44e4-b98e-a91b0b4572ea None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:04.356537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a060b-58d5-44e4-b98e-a91b0b4572ea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:04.356537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a060b-58d5-44e4-b98e-a91b0b4572ea None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:04.357921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 596/1174] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:57:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:57:04.365737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16564b5f-cedd-4530-9f77-a6fd4fa1ef27 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:04.366306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16564b5f-cedd-4530-9f77-a6fd4fa1ef27 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:04.366481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16564b5f-cedd-4530-9f77-a6fd4fa1ef27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:04.388265 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-16564b5f-cedd-4530-9f77-a6fd4fa1ef27 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:57:04.389501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-16564b5f-cedd-4530-9f77-a6fd4fa1ef27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:57:04.467089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 579/1175] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:57:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:57:04.771256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a060b-58d5-44e4-b98e-a91b0b4572ea None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:04.893492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a060b-58d5-44e4-b98e-a91b0b4572ea None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:04.932488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fa7d1-74ce-4f12-a5bd-5e1fbf398610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:04.933250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa7d1-74ce-4f12-a5bd-5e1fbf398610 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:04.933708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa7d1-74ce-4f12-a5bd-5e1fbf398610 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:04.934020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fa7d1-74ce-4f12-a5bd-5e1fbf398610 None admin] PATH_INFO: `/v3/domains/9cdf061bf3cf4acfb483512e4fea73a0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:04.935324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fa7d1-74ce-4f12-a5bd-5e1fbf398610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9cdf061bf3cf4acfb483512e4fea73a0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:04.936793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fa7d1-74ce-4f12-a5bd-5e1fbf398610 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:04.938470 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 597/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Fri Aug 19 20:57:04 2022] GET /identity/v3/domains/9cdf061bf3cf4acfb483512e4fea73a0 => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:57:04.947529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16564b5f-cedd-4530-9f77-a6fd4fa1ef27 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:05.061493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16564b5f-cedd-4530-9f77-a6fd4fa1ef27 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:05.080936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:05.103438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff50018-c2f3-45b0-8539-ed8abb89e9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:05.104104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff50018-c2f3-45b0-8539-ed8abb89e9cb None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:05.104388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff50018-c2f3-45b0-8539-ed8abb89e9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:05.104709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff50018-c2f3-45b0-8539-ed8abb89e9cb None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:05.105394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff50018-c2f3-45b0-8539-ed8abb89e9cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:05.106881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff50018-c2f3-45b0-8539-ed8abb89e9cb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:05.131109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 580/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:57:04 2022] POST /identity/v3/users => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:57:07.292851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93258d-e1a4-4cf2-9461-8d389a017631 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:07.293491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93258d-e1a4-4cf2-9461-8d389a017631 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:07.293491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b93258d-e1a4-4cf2-9461-8d389a017631 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:07.294968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 598/1178] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:57:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:57:07.302545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e59e4b-6819-4bca-8d98-951490604207 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:07.303257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e59e4b-6819-4bca-8d98-951490604207 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:07.303503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e59e4b-6819-4bca-8d98-951490604207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:07.331495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-36e59e4b-6819-4bca-8d98-951490604207 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:57:07.333626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-36e59e4b-6819-4bca-8d98-951490604207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:57:07.430546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 581/1179] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:57:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:57:07.715362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93258d-e1a4-4cf2-9461-8d389a017631 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:07.790691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:07.841836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b93258d-e1a4-4cf2-9461-8d389a017631 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:07.898674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c76892-43a7-41ce-8c73-c6b3fcd401b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:07.899730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c76892-43a7-41ce-8c73-c6b3fcd401b9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:07.900201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c76892-43a7-41ce-8c73-c6b3fcd401b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:07.900712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c76892-43a7-41ce-8c73-c6b3fcd401b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:07.922278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c76892-43a7-41ce-8c73-c6b3fcd401b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-82c76892-43a7-41ce-8c73-c6b3fcd401b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:57:07.931727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:57:07.933621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:57:07.934675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:57:07.935441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 599/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:57:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:57:07.940998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e59e4b-6819-4bca-8d98-951490604207 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:08.104466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e59e4b-6819-4bca-8d98-951490604207 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:08.107326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:08.155178 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3a1e9-a5dc-47fd-9f3a-6f0704be4b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:08.155825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3a1e9-a5dc-47fd-9f3a-6f0704be4b40 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:08.156104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3a1e9-a5dc-47fd-9f3a-6f0704be4b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:08.156435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3a1e9-a5dc-47fd-9f3a-6f0704be4b40 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:08.157197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3a1e9-a5dc-47fd-9f3a-6f0704be4b40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:57:08.157569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3a1e9-a5dc-47fd-9f3a-6f0704be4b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:08.159244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3a1e9-a5dc-47fd-9f3a-6f0704be4b40 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:08.169884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 582/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:57:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:57:08.175990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c76892-43a7-41ce-8c73-c6b3fcd401b9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:08.294513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c76892-43a7-41ce-8c73-c6b3fcd401b9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:08.314481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:08.333714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3aac5-22a1-4c2c-902a-0d7b9916b01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:08.334383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3aac5-22a1-4c2c-902a-0d7b9916b01f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:08.334680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3aac5-22a1-4c2c-902a-0d7b9916b01f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:08.334950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3aac5-22a1-4c2c-902a-0d7b9916b01f None admin] PATH_INFO: `/v3/users/df8545a63279494f9e13cf05eec1fe82` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:08.350671 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3aac5-22a1-4c2c-902a-0d7b9916b01f None admin] RBAC: Authorizing `identity:get_user(user_id=df8545a63279494f9e13cf05eec1fe82)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:08.352042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3aac5-22a1-4c2c-902a-0d7b9916b01f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:08.353746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 600/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:57:08 2022] GET /identity/v3/users/df8545a63279494f9e13cf05eec1fe82 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:57:08.360525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3a1e9-a5dc-47fd-9f3a-6f0704be4b40 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:08.477528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3a1e9-a5dc-47fd-9f3a-6f0704be4b40 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:08.525946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e685e9-4322-4699-b79a-3dd873561fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:08.527460 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e685e9-4322-4699-b79a-3dd873561fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:08.527803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e685e9-4322-4699-b79a-3dd873561fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:08.528106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e685e9-4322-4699-b79a-3dd873561fd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:08.565897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:08.610564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e685e9-4322-4699-b79a-3dd873561fd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:08.611977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e685e9-4322-4699-b79a-3dd873561fd2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:08.733710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 583/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:57:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:57:08.740244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3aac5-22a1-4c2c-902a-0d7b9916b01f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:08.855210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3aac5-22a1-4c2c-902a-0d7b9916b01f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:08.896385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4b44d8-9b4e-4483-bf09-b2706614b30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:08.896921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b44d8-9b4e-4483-bf09-b2706614b30c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:08.897202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b44d8-9b4e-4483-bf09-b2706614b30c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:08.897476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4b44d8-9b4e-4483-bf09-b2706614b30c None admin] PATH_INFO: `/v3/projects/ef6194e43d4349759386b4ba35828b2c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:08.898694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b44d8-9b4e-4483-bf09-b2706614b30c None admin] RBAC: Authorizing `identity:get_project(project_id=ef6194e43d4349759386b4ba35828b2c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:08.900139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4b44d8-9b4e-4483-bf09-b2706614b30c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:08.902412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 601/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:57:08 2022] GET /identity/v3/projects/ef6194e43d4349759386b4ba35828b2c => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:57:08.909141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e685e9-4322-4699-b79a-3dd873561fd2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:08.980838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:09.024425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e685e9-4322-4699-b79a-3dd873561fd2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:09.063999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f537ef5-15eb-4329-be8c-6d719eeaa154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:09.064606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f537ef5-15eb-4329-be8c-6d719eeaa154 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:09.064883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f537ef5-15eb-4329-be8c-6d719eeaa154 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:09.065158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f537ef5-15eb-4329-be8c-6d719eeaa154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:09.065915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f537ef5-15eb-4329-be8c-6d719eeaa154 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=ef6194e43d4349759386b4ba35828b2c, user.id=df8545a63279494f9e13cf05eec1fe82) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:57:09.066228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f537ef5-15eb-4329-be8c-6d719eeaa154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:09.067809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f537ef5-15eb-4329-be8c-6d719eeaa154 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:09.074884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 584/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:57:08 2022] GET /identity/v3/role_assignments?user.id=df8545a63279494f9e13cf05eec1fe82&scope.project.id=ef6194e43d4349759386b4ba35828b2c&role.id=8b350845f331423cb0494e3449b58310 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:57:11.037095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662272f-f4c3-4ad7-8b81-0b5ada08218d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:11.037699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662272f-f4c3-4ad7-8b81-0b5ada08218d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:11.037699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662272f-f4c3-4ad7-8b81-0b5ada08218d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:11.039133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 602/1186] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:57:11.045197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23253b58-58b9-469f-8965-02dd95637dfc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:11.045691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23253b58-58b9-469f-8965-02dd95637dfc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:11.046190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23253b58-58b9-469f-8965-02dd95637dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:11.079523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-23253b58-58b9-469f-8965-02dd95637dfc None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:57:11.080959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-23253b58-58b9-469f-8965-02dd95637dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:57:11.200658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 585/1187] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:57:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:57:11.479175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662272f-f4c3-4ad7-8b81-0b5ada08218d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:11.535276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:11.606189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662272f-f4c3-4ad7-8b81-0b5ada08218d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:11.644874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812f98-b7a1-46aa-9565-faa57b1b0741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:11.645519 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812f98-b7a1-46aa-9565-faa57b1b0741 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:11.645827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812f98-b7a1-46aa-9565-faa57b1b0741 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:11.646103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6812f98-b7a1-46aa-9565-faa57b1b0741 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:11.657852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6812f98-b7a1-46aa-9565-faa57b1b0741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d6812f98-b7a1-46aa-9565-faa57b1b0741 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:57:11.662741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:57:11.664463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:57:11.666334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 603/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:57:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:57:11.672594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23253b58-58b9-469f-8965-02dd95637dfc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:11.758474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:11.837345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23253b58-58b9-469f-8965-02dd95637dfc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:11.908415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4780609-40ea-4d90-b6e7-e7f3c63bd410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:11.909488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4780609-40ea-4d90-b6e7-e7f3c63bd410 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:11.910164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4780609-40ea-4d90-b6e7-e7f3c63bd410 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:11.910853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4780609-40ea-4d90-b6e7-e7f3c63bd410 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:11.912206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4780609-40ea-4d90-b6e7-e7f3c63bd410 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:57:11.912763 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4780609-40ea-4d90-b6e7-e7f3c63bd410 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:11.915546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4780609-40ea-4d90-b6e7-e7f3c63bd410 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:11.932499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 586/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:57:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:57:11.939991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812f98-b7a1-46aa-9565-faa57b1b0741 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:12.065550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6812f98-b7a1-46aa-9565-faa57b1b0741 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:12.067981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:12.108551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfffbf-e1c9-4d91-af24-a95da1d41f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:12.109219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfffbf-e1c9-4d91-af24-a95da1d41f4b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:12.109501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfffbf-e1c9-4d91-af24-a95da1d41f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:12.109934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dfffbf-e1c9-4d91-af24-a95da1d41f4b None admin] PATH_INFO: `/v3/users/df8545a63279494f9e13cf05eec1fe82` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:12.111216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfffbf-e1c9-4d91-af24-a95da1d41f4b None admin] RBAC: Authorizing `identity:get_user(user_id=df8545a63279494f9e13cf05eec1fe82)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:12.113223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dfffbf-e1c9-4d91-af24-a95da1d41f4b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:12.115090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 604/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:57:11 2022] GET /identity/v3/users/df8545a63279494f9e13cf05eec1fe82 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:57:12.121968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4780609-40ea-4d90-b6e7-e7f3c63bd410 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:12.231525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4780609-40ea-4d90-b6e7-e7f3c63bd410 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:12.262537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:12.278175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c7bb39-36bd-48ea-b934-f490a1140ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:12.278821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7bb39-36bd-48ea-b934-f490a1140ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:12.279106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7bb39-36bd-48ea-b934-f490a1140ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:12.279381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c7bb39-36bd-48ea-b934-f490a1140ec4 None admin] PATH_INFO: `/v3/projects/ef6194e43d4349759386b4ba35828b2c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:12.280592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7bb39-36bd-48ea-b934-f490a1140ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=ef6194e43d4349759386b4ba35828b2c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:12.282074 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c7bb39-36bd-48ea-b934-f490a1140ec4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:12.283780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 587/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:57:12 2022] GET /identity/v3/projects/ef6194e43d4349759386b4ba35828b2c => generated 361 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:57:12.289938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfffbf-e1c9-4d91-af24-a95da1d41f4b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:12.394790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dfffbf-e1c9-4d91-af24-a95da1d41f4b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:12.432319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9b0078-4c0f-4f29-90e4-b69db89cdd4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:12.433734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b0078-4c0f-4f29-90e4-b69db89cdd4e None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:12.434127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b0078-4c0f-4f29-90e4-b69db89cdd4e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:12.434477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9b0078-4c0f-4f29-90e4-b69db89cdd4e None admin] PATH_INFO: `/v3/projects/ef6194e43d4349759386b4ba35828b2c/users/df8545a63279494f9e13cf05eec1fe82/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:12.436545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b0078-4c0f-4f29-90e4-b69db89cdd4e None admin] RBAC: Authorizing `identity:create_grant(project_id=ef6194e43d4349759386b4ba35828b2c, user_id=df8545a63279494f9e13cf05eec1fe82, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:12.438018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9b0078-4c0f-4f29-90e4-b69db89cdd4e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:12.452374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 605/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:57:12 2022] PUT /identity/v3/projects/ef6194e43d4349759386b4ba35828b2c/users/df8545a63279494f9e13cf05eec1fe82/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:57:14.428613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60813d20-ec19-453b-92e3-146ef39cb8c6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:14.429133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60813d20-ec19-453b-92e3-146ef39cb8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:14.429325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60813d20-ec19-453b-92e3-146ef39cb8c6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:14.431341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 588/1193] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:57:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:57:14.438197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32fe0-09ac-404c-9349-926ce94b6b42 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:14.438804 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32fe0-09ac-404c-9349-926ce94b6b42 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:14.438804 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32fe0-09ac-404c-9349-926ce94b6b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:14.458279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-dac32fe0-09ac-404c-9349-926ce94b6b42 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:57:14.459510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dac32fe0-09ac-404c-9349-926ce94b6b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:57:14.490109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:14.535922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 606/1194] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:57:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:57:14.857861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60813d20-ec19-453b-92e3-146ef39cb8c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:14.982808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60813d20-ec19-453b-92e3-146ef39cb8c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:15.026887 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3b239-5518-476b-a6b2-de9a3c1b9f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:15.027529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3b239-5518-476b-a6b2-de9a3c1b9f74 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:15.027808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3b239-5518-476b-a6b2-de9a3c1b9f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:15.028089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e3b239-5518-476b-a6b2-de9a3c1b9f74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:15.039344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e3b239-5518-476b-a6b2-de9a3c1b9f74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-40e3b239-5518-476b-a6b2-de9a3c1b9f74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:57:15.044357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:57:15.045587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:57:15.046739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 589/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:57:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:57:15.051744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac32fe0-09ac-404c-9349-926ce94b6b42 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:15.176942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac32fe0-09ac-404c-9349-926ce94b6b42 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:15.213407 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20785f-c6e7-47b9-abd2-8a20d2d57d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:15.214669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20785f-c6e7-47b9-abd2-8a20d2d57d72 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:15.214949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20785f-c6e7-47b9-abd2-8a20d2d57d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:15.215224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20785f-c6e7-47b9-abd2-8a20d2d57d72 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:15.215897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20785f-c6e7-47b9-abd2-8a20d2d57d72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:57:15.216201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20785f-c6e7-47b9-abd2-8a20d2d57d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:15.217595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20785f-c6e7-47b9-abd2-8a20d2d57d72 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:15.226093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 607/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:57:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:57:15.232631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3b239-5518-476b-a6b2-de9a3c1b9f74 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:15.288685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:15.343230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e3b239-5518-476b-a6b2-de9a3c1b9f74 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:15.378610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54254439-7753-4ab3-b566-e17c08038311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:15.379243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54254439-7753-4ab3-b566-e17c08038311 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:15.379526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54254439-7753-4ab3-b566-e17c08038311 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:15.379801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54254439-7753-4ab3-b566-e17c08038311 None admin] PATH_INFO: `/v3/users/df8545a63279494f9e13cf05eec1fe82` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:15.380949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54254439-7753-4ab3-b566-e17c08038311 None admin] RBAC: Authorizing `identity:get_user(user_id=df8545a63279494f9e13cf05eec1fe82)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:15.382436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54254439-7753-4ab3-b566-e17c08038311 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:15.384046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 590/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Fri Aug 19 20:57:15 2022] GET /identity/v3/users/df8545a63279494f9e13cf05eec1fe82 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:57:15.393125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20785f-c6e7-47b9-abd2-8a20d2d57d72 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:15.419459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:15.501569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20785f-c6e7-47b9-abd2-8a20d2d57d72 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:15.547797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc514ee7-5206-4798-9780-7191f9d76afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:15.548420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc514ee7-5206-4798-9780-7191f9d76afd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:15.548718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc514ee7-5206-4798-9780-7191f9d76afd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:15.548968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc514ee7-5206-4798-9780-7191f9d76afd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:15.622921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc514ee7-5206-4798-9780-7191f9d76afd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:15.624797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc514ee7-5206-4798-9780-7191f9d76afd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:15.650813 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:15.732148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 608/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:57:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 19 20:57:15.738340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54254439-7753-4ab3-b566-e17c08038311 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:15.855385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54254439-7753-4ab3-b566-e17c08038311 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:15.873310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:15.893204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb64598-9247-4393-90c2-e0c3e7f1aa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:15.893869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb64598-9247-4393-90c2-e0c3e7f1aa9b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:15.894151 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb64598-9247-4393-90c2-e0c3e7f1aa9b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:15.894420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb64598-9247-4393-90c2-e0c3e7f1aa9b None admin] PATH_INFO: `/v3/projects/ef6194e43d4349759386b4ba35828b2c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:15.895540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb64598-9247-4393-90c2-e0c3e7f1aa9b None admin] RBAC: Authorizing `identity:get_project(project_id=ef6194e43d4349759386b4ba35828b2c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:15.896969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb64598-9247-4393-90c2-e0c3e7f1aa9b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:15.898619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 591/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Fri Aug 19 20:57:15 2022] GET /identity/v3/projects/ef6194e43d4349759386b4ba35828b2c => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:57:15.906298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc514ee7-5206-4798-9780-7191f9d76afd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:57:16.021335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc514ee7-5206-4798-9780-7191f9d76afd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:57:16.047621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:16.059555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a6e76-e434-4ea7-8470-85b2b30b4ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:57:16.060159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a6e76-e434-4ea7-8470-85b2b30b4ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:57:16.060434 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a6e76-e434-4ea7-8470-85b2b30b4ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:57:16.060698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a6e76-e434-4ea7-8470-85b2b30b4ac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:57:16.061420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a6e76-e434-4ea7-8470-85b2b30b4ac1 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=ef6194e43d4349759386b4ba35828b2c, user.id=df8545a63279494f9e13cf05eec1fe82) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:57:16.061721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a6e76-e434-4ea7-8470-85b2b30b4ac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:57:16.063181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a6e76-e434-4ea7-8470-85b2b30b4ac1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:57:16.069333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 609/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:57:15 2022] GET /identity/v3/role_assignments?user.id=df8545a63279494f9e13cf05eec1fe82&scope.project.id=ef6194e43d4349759386b4ba35828b2c&role.id=8b350845f331423cb0494e3449b58310 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:57:25.888751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:57:26.066543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:14.963818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9090a-94f1-4c8d-b128-89ada2ffab11 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:14.964557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9090a-94f1-4c8d-b128-89ada2ffab11 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:14.964557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b9090a-94f1-4c8d-b128-89ada2ffab11 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:14.966798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 592/1201] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:58:14.977708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e61cf-664f-466d-94a3-0cac24a882aa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:14.977708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e61cf-664f-466d-94a3-0cac24a882aa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:14.977708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e61cf-664f-466d-94a3-0cac24a882aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:15.003073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6b4e61cf-664f-466d-94a3-0cac24a882aa None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:15.004450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6b4e61cf-664f-466d-94a3-0cac24a882aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:15.159039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 610/1202] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:58:15.482562 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9090a-94f1-4c8d-b128-89ada2ffab11 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:15.621998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b9090a-94f1-4c8d-b128-89ada2ffab11 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:15.684216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d62ad-afd9-4284-bded-e093c08d2e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:15.685226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d62ad-afd9-4284-bded-e093c08d2e44 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:15.685696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d62ad-afd9-4284-bded-e093c08d2e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:15.686203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d62ad-afd9-4284-bded-e093c08d2e44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:15.719550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d62ad-afd9-4284-bded-e093c08d2e44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:15.722142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d62ad-afd9-4284-bded-e093c08d2e44 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-874d62ad-afd9-4284-bded-e093c08d2e44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:15.758140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:15.761985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:15.764138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:15.764138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:15.764138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:15.764138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 593/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:15.773172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e61cf-664f-466d-94a3-0cac24a882aa None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:15.917527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e61cf-664f-466d-94a3-0cac24a882aa None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:15.957329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:15.967848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58d79f-f957-4616-a5d9-c3fc76db47a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:15.968473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58d79f-f957-4616-a5d9-c3fc76db47a6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:15.968750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58d79f-f957-4616-a5d9-c3fc76db47a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:15.969042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58d79f-f957-4616-a5d9-c3fc76db47a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:15.969746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58d79f-f957-4616-a5d9-c3fc76db47a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:15.970103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58d79f-f957-4616-a5d9-c3fc76db47a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:15.971780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58d79f-f957-4616-a5d9-c3fc76db47a6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:15.992556 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 611/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:58:15.993190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d62ad-afd9-4284-bded-e093c08d2e44 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:16.071156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:16.159292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d62ad-afd9-4284-bded-e093c08d2e44 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:16.223909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac48156-9aef-4133-a5e1-70ada083a23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:16.224470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac48156-9aef-4133-a5e1-70ada083a23d None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:16.225322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac48156-9aef-4133-a5e1-70ada083a23d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:16.225322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac48156-9aef-4133-a5e1-70ada083a23d None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:16.225687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac48156-9aef-4133-a5e1-70ada083a23d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:16.227145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac48156-9aef-4133-a5e1-70ada083a23d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:16.265830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 594/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Fri Aug 19 20:58:15 2022] POST /identity/v3/users => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 20:58:18.467833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a09e0-77b2-4f55-bb89-a4ccd907998e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:18.468413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a09e0-77b2-4f55-bb89-a4ccd907998e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:18.468413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a09e0-77b2-4f55-bb89-a4ccd907998e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:18.469900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 612/1206] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:58:18.476780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1afbd-f02e-4177-8c91-8ce304dd2a6f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:18.476780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1afbd-f02e-4177-8c91-8ce304dd2a6f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:18.476780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1afbd-f02e-4177-8c91-8ce304dd2a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:18.502748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-01e1afbd-f02e-4177-8c91-8ce304dd2a6f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:18.504279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-01e1afbd-f02e-4177-8c91-8ce304dd2a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:18.613791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 595/1207] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:58:18.909388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a09e0-77b2-4f55-bb89-a4ccd907998e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:19.049511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a09e0-77b2-4f55-bb89-a4ccd907998e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:19.099743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f015f-2f71-4cb9-a506-fe5b7039451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:19.100395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f015f-2f71-4cb9-a506-fe5b7039451e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:19.100672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f015f-2f71-4cb9-a506-fe5b7039451e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:19.100951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f015f-2f71-4cb9-a506-fe5b7039451e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:19.114086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f015f-2f71-4cb9-a506-fe5b7039451e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-777f015f-2f71-4cb9-a506-fe5b7039451e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:58:19.123014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:19.124960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:19.125842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 613/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:58:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:58:19.131135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1afbd-f02e-4177-8c91-8ce304dd2a6f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:19.191009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:19.319621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1afbd-f02e-4177-8c91-8ce304dd2a6f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:19.374594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92016f72-9743-41be-a391-2fa3155a484b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:19.375322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92016f72-9743-41be-a391-2fa3155a484b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:19.375657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92016f72-9743-41be-a391-2fa3155a484b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:19.375918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92016f72-9743-41be-a391-2fa3155a484b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:19.376659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92016f72-9743-41be-a391-2fa3155a484b None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:19.376997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92016f72-9743-41be-a391-2fa3155a484b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:19.378490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92016f72-9743-41be-a391-2fa3155a484b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:19.388815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 596/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:58:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:58:19.395251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f015f-2f71-4cb9-a506-fe5b7039451e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:19.463951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:19.520866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f015f-2f71-4cb9-a506-fe5b7039451e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:19.565670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5e40f-6f88-4cfb-90f5-610cba8a506b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:19.566342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5e40f-6f88-4cfb-90f5-610cba8a506b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:19.566624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5e40f-6f88-4cfb-90f5-610cba8a506b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:19.566900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5e40f-6f88-4cfb-90f5-610cba8a506b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:19.578658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5e40f-6f88-4cfb-90f5-610cba8a506b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:19.580279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5e40f-6f88-4cfb-90f5-610cba8a506b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b1b5e40f-6f88-4cfb-90f5-610cba8a506b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:19.597418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:19.598715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:19.599603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:19.599603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:19.599603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:19.599603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 614/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:58:19.605872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92016f72-9743-41be-a391-2fa3155a484b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:19.737657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:19.740748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92016f72-9743-41be-a391-2fa3155a484b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:19.793375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7b2ad-2cf2-40a2-920d-17aa6ed48a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:19.794027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7b2ad-2cf2-40a2-920d-17aa6ed48a9d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:19.794302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7b2ad-2cf2-40a2-920d-17aa6ed48a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:19.794577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7b2ad-2cf2-40a2-920d-17aa6ed48a9d None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:19.795234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7b2ad-2cf2-40a2-920d-17aa6ed48a9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:19.795545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7b2ad-2cf2-40a2-920d-17aa6ed48a9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:19.796956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7b2ad-2cf2-40a2-920d-17aa6ed48a9d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:19.813292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 597/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:19.819358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5e40f-6f88-4cfb-90f5-610cba8a506b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:19.961358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5e40f-6f88-4cfb-90f5-610cba8a506b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:19.961358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:20.008980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9329fd-1be5-4a57-a33f-b05b18132a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:20.011764 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9329fd-1be5-4a57-a33f-b05b18132a19 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:20.011764 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9329fd-1be5-4a57-a33f-b05b18132a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:20.011764 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9329fd-1be5-4a57-a33f-b05b18132a19 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:20.020459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9329fd-1be5-4a57-a33f-b05b18132a19 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:20.022081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9329fd-1be5-4a57-a33f-b05b18132a19 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-0a9329fd-1be5-4a57-a33f-b05b18132a19 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:58:20.038213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:20.041498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:20.043535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 615/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:20.046544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7b2ad-2cf2-40a2-920d-17aa6ed48a9d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:20.188666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7b2ad-2cf2-40a2-920d-17aa6ed48a9d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:20.241696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5fdad-43e4-4b9f-8104-3de9f3197814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:20.242382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5fdad-43e4-4b9f-8104-3de9f3197814 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:20.242667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5fdad-43e4-4b9f-8104-3de9f3197814 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:20.242955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5fdad-43e4-4b9f-8104-3de9f3197814 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:20.243772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5fdad-43e4-4b9f-8104-3de9f3197814 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:20.244526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5fdad-43e4-4b9f-8104-3de9f3197814 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:20.246078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5fdad-43e4-4b9f-8104-3de9f3197814 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:20.273359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 598/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Aug 19 20:58:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:20.274777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9329fd-1be5-4a57-a33f-b05b18132a19 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:20.409603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9329fd-1be5-4a57-a33f-b05b18132a19 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:20.467934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89498653-94f7-429f-bd09-b0048e048522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:20.469113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89498653-94f7-429f-bd09-b0048e048522 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:20.469113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89498653-94f7-429f-bd09-b0048e048522 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:20.469620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89498653-94f7-429f-bd09-b0048e048522 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:20.520547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:20.574837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89498653-94f7-429f-bd09-b0048e048522 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:20.576447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89498653-94f7-429f-bd09-b0048e048522 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:20.708242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 616/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:58:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:58:20.714971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5fdad-43e4-4b9f-8104-3de9f3197814 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:20.740797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:20.852689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5fdad-43e4-4b9f-8104-3de9f3197814 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:20.897662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e109e4-a860-4ed6-bbf1-33c80dd56bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:20.898480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e109e4-a860-4ed6-bbf1-33c80dd56bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:20.898846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e109e4-a860-4ed6-bbf1-33c80dd56bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:20.899181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e109e4-a860-4ed6-bbf1-33c80dd56bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:20.910196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e109e4-a860-4ed6-bbf1-33c80dd56bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:20.911727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e109e4-a860-4ed6-bbf1-33c80dd56bc8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-12e109e4-a860-4ed6-bbf1-33c80dd56bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:20.932346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:20.933386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:20.934314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:20.934314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:20.934314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:20.938081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 599/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:20.940075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89498653-94f7-429f-bd09-b0048e048522 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:21.017157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:21.066546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89498653-94f7-429f-bd09-b0048e048522 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:21.117706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ba43c-bbdc-4427-abd4-0178654be3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:21.118255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ba43c-bbdc-4427-abd4-0178654be3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:21.118680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ba43c-bbdc-4427-abd4-0178654be3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:21.118986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ba43c-bbdc-4427-abd4-0178654be3b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:21.119599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ba43c-bbdc-4427-abd4-0178654be3b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:21.119902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ba43c-bbdc-4427-abd4-0178654be3b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:21.121545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ba43c-bbdc-4427-abd4-0178654be3b5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:21.136599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 617/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:21.143025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e109e4-a860-4ed6-bbf1-33c80dd56bc8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:21.232838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:21.285728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e109e4-a860-4ed6-bbf1-33c80dd56bc8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:21.331242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebebf4a-f239-4667-9676-072950815eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:21.332534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebebf4a-f239-4667-9676-072950815eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:21.332926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebebf4a-f239-4667-9676-072950815eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:21.333300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebebf4a-f239-4667-9676-072950815eb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:21.346471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebebf4a-f239-4667-9676-072950815eb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:21.348433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebebf4a-f239-4667-9676-072950815eb2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2ebebf4a-f239-4667-9676-072950815eb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:58:21.364697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:21.365693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:21.366647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:58:21.366647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:58:21.366647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:58:21.366647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:58:21.366647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:58:21.366647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:58:21.366647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:21.366647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:21.366647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 600/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:58:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:21.373585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ba43c-bbdc-4427-abd4-0178654be3b5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:21.491005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ba43c-bbdc-4427-abd4-0178654be3b5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:21.518181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:21.531426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6923e8dd-b4ce-42da-b9b5-5410875a5e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:21.532110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e8dd-b4ce-42da-b9b5-5410875a5e94 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:21.532402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e8dd-b4ce-42da-b9b5-5410875a5e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:21.532745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6923e8dd-b4ce-42da-b9b5-5410875a5e94 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:21.533531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923e8dd-b4ce-42da-b9b5-5410875a5e94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:21.533896 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923e8dd-b4ce-42da-b9b5-5410875a5e94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:21.535451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6923e8dd-b4ce-42da-b9b5-5410875a5e94 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:21.552587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 618/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:58:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:21.558575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebebf4a-f239-4667-9676-072950815eb2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:21.696103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebebf4a-f239-4667-9676-072950815eb2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:21.732966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:21.740191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d036edd-b938-48da-a924-64d3eacada88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:21.740957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036edd-b938-48da-a924-64d3eacada88 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:21.741271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036edd-b938-48da-a924-64d3eacada88 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:21.741516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d036edd-b938-48da-a924-64d3eacada88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:21.742511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036edd-b938-48da-a924-64d3eacada88 None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=824165e55d2f4742aa435f1edff9b403) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:21.742926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036edd-b938-48da-a924-64d3eacada88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:21.745687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d036edd-b938-48da-a924-64d3eacada88 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:21.756045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 601/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:58:21 2022] GET /identity/v3/role_assignments?user.id=824165e55d2f4742aa435f1edff9b403&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:23.680455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b4a83-edba-430c-a711-f968af634bb2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:23.681069 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b4a83-edba-430c-a711-f968af634bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:23.681069 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b4a83-edba-430c-a711-f968af634bb2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:23.682782 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 619/1220] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:58:23.688375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448478a-d17e-473e-a3a2-59f6df537e7a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:23.688841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448478a-d17e-473e-a3a2-59f6df537e7a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:23.688927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448478a-d17e-473e-a3a2-59f6df537e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:23.707977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4448478a-d17e-473e-a3a2-59f6df537e7a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:23.709181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4448478a-d17e-473e-a3a2-59f6df537e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:23.793559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 602/1221] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:58:24.084320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b4a83-edba-430c-a711-f968af634bb2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:24.201330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129b4a83-edba-430c-a711-f968af634bb2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:24.238759 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c74dbd-5326-49f4-a9ac-c1703db970ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:24.239402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c74dbd-5326-49f4-a9ac-c1703db970ca None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:24.239682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c74dbd-5326-49f4-a9ac-c1703db970ca None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:24.239949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c74dbd-5326-49f4-a9ac-c1703db970ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:24.251710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c74dbd-5326-49f4-a9ac-c1703db970ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-20c74dbd-5326-49f4-a9ac-c1703db970ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:58:24.256474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:24.257561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:24.263247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 620/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:58:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:24.264533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448478a-d17e-473e-a3a2-59f6df537e7a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:24.380696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448478a-d17e-473e-a3a2-59f6df537e7a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:24.411310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:24.423950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8c4db-8aee-4681-b0a0-92c79865772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:24.424568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8c4db-8aee-4681-b0a0-92c79865772a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:24.424871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8c4db-8aee-4681-b0a0-92c79865772a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:24.425125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8c4db-8aee-4681-b0a0-92c79865772a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:24.425831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8c4db-8aee-4681-b0a0-92c79865772a None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:24.426135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8c4db-8aee-4681-b0a0-92c79865772a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:24.427563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8c4db-8aee-4681-b0a0-92c79865772a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:24.441570 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 603/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:58:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:24.442977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c74dbd-5326-49f4-a9ac-c1703db970ca None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:24.491367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:24.554738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c74dbd-5326-49f4-a9ac-c1703db970ca None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:24.590018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c929b9b-50b3-4ffb-9fa4-3c67718d016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:24.590635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c929b9b-50b3-4ffb-9fa4-3c67718d016b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:24.590950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c929b9b-50b3-4ffb-9fa4-3c67718d016b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:24.591219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c929b9b-50b3-4ffb-9fa4-3c67718d016b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:24.601294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c929b9b-50b3-4ffb-9fa4-3c67718d016b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:24.602784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c929b9b-50b3-4ffb-9fa4-3c67718d016b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1c929b9b-50b3-4ffb-9fa4-3c67718d016b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:24.618404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:24.619463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:24.620684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:24.620684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:24.620684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:24.620684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 621/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:24.626459 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8c4db-8aee-4681-b0a0-92c79865772a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:24.733316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8c4db-8aee-4681-b0a0-92c79865772a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:24.768329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7e664-46f0-4237-a380-ac4bfd19382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:24.768974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e664-46f0-4237-a380-ac4bfd19382c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:24.769249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e664-46f0-4237-a380-ac4bfd19382c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:24.769580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7e664-46f0-4237-a380-ac4bfd19382c None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:24.770242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7e664-46f0-4237-a380-ac4bfd19382c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:24.770538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7e664-46f0-4237-a380-ac4bfd19382c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:24.772029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7e664-46f0-4237-a380-ac4bfd19382c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:24.787020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 604/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:24.794086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c929b9b-50b3-4ffb-9fa4-3c67718d016b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:24.923855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:24.931062 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c929b9b-50b3-4ffb-9fa4-3c67718d016b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:24.973936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b5df2-f72d-4b17-b8a6-3fc9260b2d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:24.974603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b5df2-f72d-4b17-b8a6-3fc9260b2d49 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:24.974860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b5df2-f72d-4b17-b8a6-3fc9260b2d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:24.975140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b5df2-f72d-4b17-b8a6-3fc9260b2d49 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:24.985568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b5df2-f72d-4b17-b8a6-3fc9260b2d49 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:24.987146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b5df2-f72d-4b17-b8a6-3fc9260b2d49 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d50b5df2-f72d-4b17-b8a6-3fc9260b2d49 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:58:25.001586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:25.003862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:25.005416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 622/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:25.008947 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7e664-46f0-4237-a380-ac4bfd19382c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:25.072948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:25.147262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7e664-46f0-4237-a380-ac4bfd19382c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:25.183964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770f469-b6d1-4fb0-9e86-f67c0fcc8ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:25.184683 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770f469-b6d1-4fb0-9e86-f67c0fcc8ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:25.184961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770f469-b6d1-4fb0-9e86-f67c0fcc8ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:25.185280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770f469-b6d1-4fb0-9e86-f67c0fcc8ca1 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:25.186247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770f469-b6d1-4fb0-9e86-f67c0fcc8ca1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:25.186589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770f469-b6d1-4fb0-9e86-f67c0fcc8ca1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:25.188305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770f469-b6d1-4fb0-9e86-f67c0fcc8ca1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:25.214666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 605/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Aug 19 20:58:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:25.216300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b5df2-f72d-4b17-b8a6-3fc9260b2d49 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:25.336779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b5df2-f72d-4b17-b8a6-3fc9260b2d49 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:25.379729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af158-9ae9-4273-9c30-0ab627a920bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:25.380368 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af158-9ae9-4273-9c30-0ab627a920bf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:25.380647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af158-9ae9-4273-9c30-0ab627a920bf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:25.380918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af158-9ae9-4273-9c30-0ab627a920bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:25.393181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00af158-9ae9-4273-9c30-0ab627a920bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:25.394809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00af158-9ae9-4273-9c30-0ab627a920bf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b00af158-9ae9-4273-9c30-0ab627a920bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:25.410240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:25.411430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:25.412553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:25.412553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:25.412553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:25.412553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 623/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:58:25.420028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770f469-b6d1-4fb0-9e86-f67c0fcc8ca1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:25.541533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770f469-b6d1-4fb0-9e86-f67c0fcc8ca1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:25.544652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:25.584064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfc6b4-9eac-4b7f-b132-477c21bc1e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:25.584705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfc6b4-9eac-4b7f-b132-477c21bc1e8c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:25.584983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfc6b4-9eac-4b7f-b132-477c21bc1e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:25.585256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfc6b4-9eac-4b7f-b132-477c21bc1e8c None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:25.586051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfc6b4-9eac-4b7f-b132-477c21bc1e8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:25.586360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfc6b4-9eac-4b7f-b132-477c21bc1e8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:25.589585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfc6b4-9eac-4b7f-b132-477c21bc1e8c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:25.603324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 606/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:58:25.608928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af158-9ae9-4273-9c30-0ab627a920bf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:25.622249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:25.744303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af158-9ae9-4273-9c30-0ab627a920bf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:25.795154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340a3fb-9aeb-4c86-90dc-74b3fad72d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:25.796124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340a3fb-9aeb-4c86-90dc-74b3fad72d76 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:25.796124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340a3fb-9aeb-4c86-90dc-74b3fad72d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:25.796323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340a3fb-9aeb-4c86-90dc-74b3fad72d76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:25.803880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340a3fb-9aeb-4c86-90dc-74b3fad72d76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:25.805380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340a3fb-9aeb-4c86-90dc-74b3fad72d76 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3340a3fb-9aeb-4c86-90dc-74b3fad72d76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:58:25.818400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:25.820198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:25.822761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:58:25.822761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:58:25.822761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:58:25.822761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:58:25.822761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:58:25.822761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:58:25.822761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:25.822761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:25.822761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 624/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:58:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:25.826401 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfc6b4-9eac-4b7f-b132-477c21bc1e8c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:25.959538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfc6b4-9eac-4b7f-b132-477c21bc1e8c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:26.004134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d9976-5e9e-45b7-ae95-53215026968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:26.004754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d9976-5e9e-45b7-ae95-53215026968c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:26.005032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d9976-5e9e-45b7-ae95-53215026968c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:26.005306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d9976-5e9e-45b7-ae95-53215026968c None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:26.006039 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d9976-5e9e-45b7-ae95-53215026968c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:26.006340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d9976-5e9e-45b7-ae95-53215026968c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:26.007782 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d9976-5e9e-45b7-ae95-53215026968c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:26.023136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 607/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:58:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:58:26.030265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340a3fb-9aeb-4c86-90dc-74b3fad72d76 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:26.110917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:26.167551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340a3fb-9aeb-4c86-90dc-74b3fad72d76 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:26.212223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c964b5-820f-4796-84bb-ce5f15cb28ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:26.212934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c964b5-820f-4796-84bb-ce5f15cb28ee None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:26.213213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c964b5-820f-4796-84bb-ce5f15cb28ee None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:26.213483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c964b5-820f-4796-84bb-ce5f15cb28ee None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/824165e55d2f4742aa435f1edff9b403/roles/a797bb15d18b42148f36a77bd6d59606` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:26.229684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c964b5-820f-4796-84bb-ce5f15cb28ee None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=824165e55d2f4742aa435f1edff9b403, role_id=a797bb15d18b42148f36a77bd6d59606)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:26.231374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c964b5-820f-4796-84bb-ce5f15cb28ee None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:26.250151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 625/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:58:26 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/824165e55d2f4742aa435f1edff9b403/roles/a797bb15d18b42148f36a77bd6d59606 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:58:28.342464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac282694-0ffd-4e62-9d70-5c5408cfa558 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:28.342967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac282694-0ffd-4e62-9d70-5c5408cfa558 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:28.342967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac282694-0ffd-4e62-9d70-5c5408cfa558 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:28.344269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 608/1233] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:58:28.349969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48719-4db9-44eb-bf96-faa26b4c4175 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:28.350683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48719-4db9-44eb-bf96-faa26b4c4175 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:28.350683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48719-4db9-44eb-bf96-faa26b4c4175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:28.374892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5ab48719-4db9-44eb-bf96-faa26b4c4175 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:28.377473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5ab48719-4db9-44eb-bf96-faa26b4c4175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:28.457439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 626/1234] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:58:28.740002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac282694-0ffd-4e62-9d70-5c5408cfa558 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:28.768360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:28.853786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac282694-0ffd-4e62-9d70-5c5408cfa558 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:28.894917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0eaf7-49cc-41e6-8e0a-249ca01dc9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:28.895563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0eaf7-49cc-41e6-8e0a-249ca01dc9ab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:28.895844 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0eaf7-49cc-41e6-8e0a-249ca01dc9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:28.896122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0eaf7-49cc-41e6-8e0a-249ca01dc9ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:28.908069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0eaf7-49cc-41e6-8e0a-249ca01dc9ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-26e0eaf7-49cc-41e6-8e0a-249ca01dc9ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:58:28.913566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:28.914730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:28.918968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 609/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Fri Aug 19 20:58:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:28.921787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48719-4db9-44eb-bf96-faa26b4c4175 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:28.975667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:29.045938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48719-4db9-44eb-bf96-faa26b4c4175 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:29.096080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4e7b2-9ebe-476d-b55e-dc927ec6e4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:29.096691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4e7b2-9ebe-476d-b55e-dc927ec6e4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:29.096970 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4e7b2-9ebe-476d-b55e-dc927ec6e4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:29.097265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4e7b2-9ebe-476d-b55e-dc927ec6e4f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:29.097941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4e7b2-9ebe-476d-b55e-dc927ec6e4f7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:29.098244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4e7b2-9ebe-476d-b55e-dc927ec6e4f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:29.099611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4e7b2-9ebe-476d-b55e-dc927ec6e4f7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:29.108954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 627/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Fri Aug 19 20:58:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:29.115001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0eaf7-49cc-41e6-8e0a-249ca01dc9ab None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:29.211837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:29.232580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0eaf7-49cc-41e6-8e0a-249ca01dc9ab None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:29.275327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a0d47-27e4-4066-9035-3721e0c431b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:29.276335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a0d47-27e4-4066-9035-3721e0c431b7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:29.276897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a0d47-27e4-4066-9035-3721e0c431b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:29.277397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a0d47-27e4-4066-9035-3721e0c431b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:29.293792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a0d47-27e4-4066-9035-3721e0c431b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:29.295495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a0d47-27e4-4066-9035-3721e0c431b7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9c5a0d47-27e4-4066-9035-3721e0c431b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:29.313787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:29.318902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:29.320010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:29.320010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:29.320010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:29.320010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 610/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:58:29.323198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4e7b2-9ebe-476d-b55e-dc927ec6e4f7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:29.459649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4e7b2-9ebe-476d-b55e-dc927ec6e4f7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:29.478262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:29.508891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96496825-2ead-49b1-bcc6-9fd52e0ed556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:29.510382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96496825-2ead-49b1-bcc6-9fd52e0ed556 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:29.510709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96496825-2ead-49b1-bcc6-9fd52e0ed556 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:29.510980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96496825-2ead-49b1-bcc6-9fd52e0ed556 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:29.511632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96496825-2ead-49b1-bcc6-9fd52e0ed556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:29.511933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96496825-2ead-49b1-bcc6-9fd52e0ed556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:29.513302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96496825-2ead-49b1-bcc6-9fd52e0ed556 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:29.534753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 628/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:29.541472 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a0d47-27e4-4066-9035-3721e0c431b7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:29.678527 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a0d47-27e4-4066-9035-3721e0c431b7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:29.720099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265e3b6-8dda-42cb-a6af-8344278e3051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:29.720848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265e3b6-8dda-42cb-a6af-8344278e3051 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:29.721239 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265e3b6-8dda-42cb-a6af-8344278e3051 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:29.721651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265e3b6-8dda-42cb-a6af-8344278e3051 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:29.731372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:29.733256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265e3b6-8dda-42cb-a6af-8344278e3051 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:29.735778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265e3b6-8dda-42cb-a6af-8344278e3051 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f265e3b6-8dda-42cb-a6af-8344278e3051 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:58:29.752412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:29.754600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:29.756816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 611/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:29.764007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96496825-2ead-49b1-bcc6-9fd52e0ed556 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:29.907565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96496825-2ead-49b1-bcc6-9fd52e0ed556 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:29.951968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22afb7-722a-49f3-a5c4-be4025d90601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:29.952571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22afb7-722a-49f3-a5c4-be4025d90601 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:29.952867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22afb7-722a-49f3-a5c4-be4025d90601 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:29.953134 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22afb7-722a-49f3-a5c4-be4025d90601 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:29.953915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22afb7-722a-49f3-a5c4-be4025d90601 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:29.954221 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22afb7-722a-49f3-a5c4-be4025d90601 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:29.955870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22afb7-722a-49f3-a5c4-be4025d90601 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:29.975995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 629/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Aug 19 20:58:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:58:29.986824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265e3b6-8dda-42cb-a6af-8344278e3051 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:30.142475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265e3b6-8dda-42cb-a6af-8344278e3051 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:30.189023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04528f51-a107-4166-8fb1-ca76b813f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:30.189936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04528f51-a107-4166-8fb1-ca76b813f59c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:30.190277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04528f51-a107-4166-8fb1-ca76b813f59c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:30.190627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04528f51-a107-4166-8fb1-ca76b813f59c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:30.279391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:30.290702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04528f51-a107-4166-8fb1-ca76b813f59c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:30.292219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04528f51-a107-4166-8fb1-ca76b813f59c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:30.433052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 612/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:58:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:58:30.440362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22afb7-722a-49f3-a5c4-be4025d90601 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:30.479495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:30.597355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22afb7-722a-49f3-a5c4-be4025d90601 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:30.645049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d0e57-5060-4ec2-889e-63065874d5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:30.645688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d0e57-5060-4ec2-889e-63065874d5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:30.646105 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d0e57-5060-4ec2-889e-63065874d5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:30.646431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d0e57-5060-4ec2-889e-63065874d5e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:30.660846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d0e57-5060-4ec2-889e-63065874d5e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:30.662449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d0e57-5060-4ec2-889e-63065874d5e2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9e7d0e57-5060-4ec2-889e-63065874d5e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:30.681718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:30.683494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:30.685106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:30.685106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:30.685106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:30.685106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 630/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:30.689282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04528f51-a107-4166-8fb1-ca76b813f59c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:30.812380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:30.820799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04528f51-a107-4166-8fb1-ca76b813f59c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:30.880222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813177a-b473-43fa-a459-da2e5d1c546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:30.880598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813177a-b473-43fa-a459-da2e5d1c546d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:30.880889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813177a-b473-43fa-a459-da2e5d1c546d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:30.881168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813177a-b473-43fa-a459-da2e5d1c546d None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:30.881952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813177a-b473-43fa-a459-da2e5d1c546d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:30.882271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813177a-b473-43fa-a459-da2e5d1c546d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:30.883669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813177a-b473-43fa-a459-da2e5d1c546d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:30.896968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 613/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:30.903937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d0e57-5060-4ec2-889e-63065874d5e2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:31.012187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:31.039030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d0e57-5060-4ec2-889e-63065874d5e2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:31.082284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375246e-aec1-4eb5-a14a-1c510ef1cbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:31.082940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375246e-aec1-4eb5-a14a-1c510ef1cbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:31.083263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375246e-aec1-4eb5-a14a-1c510ef1cbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:31.083469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375246e-aec1-4eb5-a14a-1c510ef1cbf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:31.091772 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375246e-aec1-4eb5-a14a-1c510ef1cbf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:31.093244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375246e-aec1-4eb5-a14a-1c510ef1cbf3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e375246e-aec1-4eb5-a14a-1c510ef1cbf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:58:31.105495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:31.106689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:31.108290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:58:31.108290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:58:31.108290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:58:31.108290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:58:31.108290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:58:31.108290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:58:31.108290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:31.108290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:31.108290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 631/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:58:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:31.112834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813177a-b473-43fa-a459-da2e5d1c546d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:31.230578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813177a-b473-43fa-a459-da2e5d1c546d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:31.268600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd43bf0-867e-46c5-870f-cbf0a6b81967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:31.269211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43bf0-867e-46c5-870f-cbf0a6b81967 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:31.269514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43bf0-867e-46c5-870f-cbf0a6b81967 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:31.269806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43bf0-867e-46c5-870f-cbf0a6b81967 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:31.270524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd43bf0-867e-46c5-870f-cbf0a6b81967 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:31.270892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd43bf0-867e-46c5-870f-cbf0a6b81967 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:31.272268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd43bf0-867e-46c5-870f-cbf0a6b81967 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:31.286030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 614/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:58:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:31.293064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375246e-aec1-4eb5-a14a-1c510ef1cbf3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:31.411207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375246e-aec1-4eb5-a14a-1c510ef1cbf3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:31.451382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcffaf-cbce-4842-831b-5cb117141230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:31.451927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcffaf-cbce-4842-831b-5cb117141230 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:31.452359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcffaf-cbce-4842-831b-5cb117141230 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:31.452539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcffaf-cbce-4842-831b-5cb117141230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:31.453299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcffaf-cbce-4842-831b-5cb117141230 None admin] RBAC: Adding query filter params (role.id=a797bb15d18b42148f36a77bd6d59606, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=824165e55d2f4742aa435f1edff9b403) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:31.453650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcffaf-cbce-4842-831b-5cb117141230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:31.455217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcffaf-cbce-4842-831b-5cb117141230 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:31.461860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 632/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:58:31 2022] GET /identity/v3/role_assignments?user.id=824165e55d2f4742aa435f1edff9b403&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=a797bb15d18b42148f36a77bd6d59606 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:33.658760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd8f67-8dc8-4641-9994-7aea2091d4e7 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:33.659378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd8f67-8dc8-4641-9994-7aea2091d4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:33.659378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd8f67-8dc8-4641-9994-7aea2091d4e7 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:33.660631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 615/1247] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:58:33.668787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af939481-5bc6-440e-84aa-f929063edee1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:33.669332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af939481-5bc6-440e-84aa-f929063edee1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:33.669867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af939481-5bc6-440e-84aa-f929063edee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:33.677631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:33.694684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-af939481-5bc6-440e-84aa-f929063edee1 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:33.696049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-af939481-5bc6-440e-84aa-f929063edee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:33.786943 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 633/1248] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:58:34.097368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd8f67-8dc8-4641-9994-7aea2091d4e7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:34.119415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:34.226589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd8f67-8dc8-4641-9994-7aea2091d4e7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:34.270172 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cec28b-2949-47eb-9173-62bc675c2621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:34.270930 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cec28b-2949-47eb-9173-62bc675c2621 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:34.271288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cec28b-2949-47eb-9173-62bc675c2621 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:34.271704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cec28b-2949-47eb-9173-62bc675c2621 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:34.287548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cec28b-2949-47eb-9173-62bc675c2621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b3cec28b-2949-47eb-9173-62bc675c2621 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:58:34.294779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:34.296110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:34.297069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 616/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:58:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:34.302974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af939481-5bc6-440e-84aa-f929063edee1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:34.416186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af939481-5bc6-440e-84aa-f929063edee1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:34.453318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:34.461013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a5231-8365-421a-9c93-47dadd32def1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:34.461777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a5231-8365-421a-9c93-47dadd32def1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:34.462098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a5231-8365-421a-9c93-47dadd32def1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:34.462498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a5231-8365-421a-9c93-47dadd32def1 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:34.463180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a5231-8365-421a-9c93-47dadd32def1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:34.463487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a5231-8365-421a-9c93-47dadd32def1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:34.464861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a5231-8365-421a-9c93-47dadd32def1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:34.482203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 634/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:58:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:58:34.483441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cec28b-2949-47eb-9173-62bc675c2621 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:34.578829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:34.605598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cec28b-2949-47eb-9173-62bc675c2621 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:34.662852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20595574-8be6-46a4-9c05-048dd9ad11cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:34.663586 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20595574-8be6-46a4-9c05-048dd9ad11cc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:34.663768 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20595574-8be6-46a4-9c05-048dd9ad11cc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:34.664041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20595574-8be6-46a4-9c05-048dd9ad11cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:34.679186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20595574-8be6-46a4-9c05-048dd9ad11cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:34.682419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20595574-8be6-46a4-9c05-048dd9ad11cc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-20595574-8be6-46a4-9c05-048dd9ad11cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:34.703725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:34.706100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:34.707902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:34.707902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:34.707902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:34.707902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 617/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:34.711300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a5231-8365-421a-9c93-47dadd32def1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:34.842570 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a5231-8365-421a-9c93-47dadd32def1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:34.882473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bcd29-581d-4d4d-bd8e-401db6af5f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:34.883067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bcd29-581d-4d4d-bd8e-401db6af5f5d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:34.883360 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bcd29-581d-4d4d-bd8e-401db6af5f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:34.883624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bcd29-581d-4d4d-bd8e-401db6af5f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:34.884921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bcd29-581d-4d4d-bd8e-401db6af5f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:34.885230 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bcd29-581d-4d4d-bd8e-401db6af5f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:34.886698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bcd29-581d-4d4d-bd8e-401db6af5f5d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:34.899387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 635/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:34.905202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20595574-8be6-46a4-9c05-048dd9ad11cc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:35.022763 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20595574-8be6-46a4-9c05-048dd9ad11cc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:35.052785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:35.061633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90205150-63dc-49e1-9f15-669d8674c35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:35.062323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90205150-63dc-49e1-9f15-669d8674c35a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:35.062606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90205150-63dc-49e1-9f15-669d8674c35a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:35.062883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90205150-63dc-49e1-9f15-669d8674c35a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:35.074567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90205150-63dc-49e1-9f15-669d8674c35a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:35.076068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90205150-63dc-49e1-9f15-669d8674c35a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-90205150-63dc-49e1-9f15-669d8674c35a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:58:35.089031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:35.091203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:35.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 618/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 19 20:58:35.097183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bcd29-581d-4d4d-bd8e-401db6af5f5d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:35.175505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:35.240887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bcd29-581d-4d4d-bd8e-401db6af5f5d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:35.284218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c46748-4d88-4513-9321-e9f59f3273af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:35.284726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c46748-4d88-4513-9321-e9f59f3273af None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:35.285003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c46748-4d88-4513-9321-e9f59f3273af None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:35.285284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c46748-4d88-4513-9321-e9f59f3273af None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:35.286042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c46748-4d88-4513-9321-e9f59f3273af None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:35.286347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c46748-4d88-4513-9321-e9f59f3273af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:35.287795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c46748-4d88-4513-9321-e9f59f3273af None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:35.312990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 636/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Aug 19 20:58:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:35.315176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90205150-63dc-49e1-9f15-669d8674c35a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:35.442101 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90205150-63dc-49e1-9f15-669d8674c35a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:35.481329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e29b41-bf3b-47ae-96f4-83473915c778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:35.481978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e29b41-bf3b-47ae-96f4-83473915c778 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:35.482262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e29b41-bf3b-47ae-96f4-83473915c778 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:35.482577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e29b41-bf3b-47ae-96f4-83473915c778 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:35.551820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:35.568976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e29b41-bf3b-47ae-96f4-83473915c778 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:35.570776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e29b41-bf3b-47ae-96f4-83473915c778 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:35.711186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 619/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:58:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:58:35.718115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c46748-4d88-4513-9321-e9f59f3273af None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:35.852632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c46748-4d88-4513-9321-e9f59f3273af None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:35.872282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:35.897349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f11e9-b796-4a99-8c3c-9ca923287a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:35.898007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f11e9-b796-4a99-8c3c-9ca923287a22 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:35.898288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f11e9-b796-4a99-8c3c-9ca923287a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:35.898576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f11e9-b796-4a99-8c3c-9ca923287a22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:35.910195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f11e9-b796-4a99-8c3c-9ca923287a22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:35.911878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952f11e9-b796-4a99-8c3c-9ca923287a22 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-952f11e9-b796-4a99-8c3c-9ca923287a22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:35.929560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:35.930923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:35.931830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:35.931830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:35.931830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:35.937805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 637/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:35.937978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e29b41-bf3b-47ae-96f4-83473915c778 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:36.037976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:36.053337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e29b41-bf3b-47ae-96f4-83473915c778 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:36.095311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62effad8-63ce-4352-b70b-673f20964d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:36.095997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62effad8-63ce-4352-b70b-673f20964d20 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:36.096284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62effad8-63ce-4352-b70b-673f20964d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:36.096680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62effad8-63ce-4352-b70b-673f20964d20 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:36.097364 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62effad8-63ce-4352-b70b-673f20964d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:36.097741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62effad8-63ce-4352-b70b-673f20964d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:36.099328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62effad8-63ce-4352-b70b-673f20964d20 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:36.118739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 620/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:36.120172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f11e9-b796-4a99-8c3c-9ca923287a22 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:36.231971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f11e9-b796-4a99-8c3c-9ca923287a22 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:36.276206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aec141-03b0-46d6-b5cf-d10c26fbd324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:36.276914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aec141-03b0-46d6-b5cf-d10c26fbd324 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:36.277319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aec141-03b0-46d6-b5cf-d10c26fbd324 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:36.277594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aec141-03b0-46d6-b5cf-d10c26fbd324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:36.286517 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aec141-03b0-46d6-b5cf-d10c26fbd324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:36.287980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aec141-03b0-46d6-b5cf-d10c26fbd324 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-99aec141-03b0-46d6-b5cf-d10c26fbd324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:58:36.303524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:36.304827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:36.306235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:58:36.306235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:58:36.306235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:58:36.306235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:58:36.306235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:58:36.306235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:58:36.306235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:36.306235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:36.310361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 638/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:36.311869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62effad8-63ce-4352-b70b-673f20964d20 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:36.435576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62effad8-63ce-4352-b70b-673f20964d20 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:36.479758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68171da2-0f9e-4742-be90-5883c982daba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:36.480467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68171da2-0f9e-4742-be90-5883c982daba None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:36.480745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68171da2-0f9e-4742-be90-5883c982daba None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:36.482176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68171da2-0f9e-4742-be90-5883c982daba None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:36.483005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68171da2-0f9e-4742-be90-5883c982daba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:36.483605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68171da2-0f9e-4742-be90-5883c982daba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:36.486955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68171da2-0f9e-4742-be90-5883c982daba None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:36.501932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 621/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:58:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:36.510741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aec141-03b0-46d6-b5cf-d10c26fbd324 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:36.543649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:36.640846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aec141-03b0-46d6-b5cf-d10c26fbd324 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:36.677589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2310f2-f1cf-455c-ac4a-546a9717e7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:36.678223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2310f2-f1cf-455c-ac4a-546a9717e7cf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:36.678497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2310f2-f1cf-455c-ac4a-546a9717e7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:36.678766 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2310f2-f1cf-455c-ac4a-546a9717e7cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:36.679488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2310f2-f1cf-455c-ac4a-546a9717e7cf None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=824165e55d2f4742aa435f1edff9b403) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:36.679795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2310f2-f1cf-455c-ac4a-546a9717e7cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:36.681866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2310f2-f1cf-455c-ac4a-546a9717e7cf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:36.687922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 639/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:58:36 2022] GET /identity/v3/role_assignments?user.id=824165e55d2f4742aa435f1edff9b403&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=8b350845f331423cb0494e3449b58310 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:38.673213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dc8b6-528e-4579-b316-906fb192a490 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:38.673788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dc8b6-528e-4579-b316-906fb192a490 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:38.673788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712dc8b6-528e-4579-b316-906fb192a490 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:38.675094 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 622/1261] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:58:38.680300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7a623-984d-4c4a-b851-bf52c466241e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:38.681229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7a623-984d-4c4a-b851-bf52c466241e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:38.681520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd7a623-984d-4c4a-b851-bf52c466241e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:38.700205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2bd7a623-984d-4c4a-b851-bf52c466241e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:38.701398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2bd7a623-984d-4c4a-b851-bf52c466241e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:38.770045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 640/1262] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:58:39.051275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712dc8b6-528e-4579-b316-906fb192a490 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:39.062234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:39.156325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712dc8b6-528e-4579-b316-906fb192a490 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:39.190031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff114065-db4f-4753-81ab-ff8c4ed518e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:39.190652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff114065-db4f-4753-81ab-ff8c4ed518e2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:39.190929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff114065-db4f-4753-81ab-ff8c4ed518e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:39.191202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff114065-db4f-4753-81ab-ff8c4ed518e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:39.201874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff114065-db4f-4753-81ab-ff8c4ed518e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ff114065-db4f-4753-81ab-ff8c4ed518e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:58:39.206524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:39.207657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:58:39.208746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:39.209931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 623/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:39.214359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7a623-984d-4c4a-b851-bf52c466241e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:39.232371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:39.312845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd7a623-984d-4c4a-b851-bf52c466241e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:39.346221 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7477fab-fe44-4b93-a646-bda4aa68b50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:39.346821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7477fab-fe44-4b93-a646-bda4aa68b50b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:39.347095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7477fab-fe44-4b93-a646-bda4aa68b50b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:39.347361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7477fab-fe44-4b93-a646-bda4aa68b50b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:39.348011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7477fab-fe44-4b93-a646-bda4aa68b50b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:39.348312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7477fab-fe44-4b93-a646-bda4aa68b50b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:39.349667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7477fab-fe44-4b93-a646-bda4aa68b50b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:39.358550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 641/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:58:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:39.364642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff114065-db4f-4753-81ab-ff8c4ed518e2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:39.432091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:39.464501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff114065-db4f-4753-81ab-ff8c4ed518e2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:39.497390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dec98-9e19-4af7-9b81-824593650d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:39.498031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dec98-9e19-4af7-9b81-824593650d76 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:39.498307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dec98-9e19-4af7-9b81-824593650d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:39.498576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539dec98-9e19-4af7-9b81-824593650d76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:39.508201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539dec98-9e19-4af7-9b81-824593650d76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:39.509662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539dec98-9e19-4af7-9b81-824593650d76 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-539dec98-9e19-4af7-9b81-824593650d76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:39.523634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:39.525201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:39.526248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:39.526248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:39.526248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:39.530095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 624/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:39.531150 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7477fab-fe44-4b93-a646-bda4aa68b50b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:39.619681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:39.641786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7477fab-fe44-4b93-a646-bda4aa68b50b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:39.678384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660c38c-a072-4e7f-a8ed-d297f6bad725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:39.678990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660c38c-a072-4e7f-a8ed-d297f6bad725 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:39.679269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660c38c-a072-4e7f-a8ed-d297f6bad725 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:39.679543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660c38c-a072-4e7f-a8ed-d297f6bad725 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:39.680190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660c38c-a072-4e7f-a8ed-d297f6bad725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:39.680497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660c38c-a072-4e7f-a8ed-d297f6bad725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:39.682528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660c38c-a072-4e7f-a8ed-d297f6bad725 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:39.694296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 642/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:39.700325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dec98-9e19-4af7-9b81-824593650d76 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:39.801420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539dec98-9e19-4af7-9b81-824593650d76 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:39.822008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:39.840830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57a988-a769-468f-b050-490b7a49ca35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:39.841627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57a988-a769-468f-b050-490b7a49ca35 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:39.841987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57a988-a769-468f-b050-490b7a49ca35 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:39.842270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57a988-a769-468f-b050-490b7a49ca35 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:39.853122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57a988-a769-468f-b050-490b7a49ca35 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:39.855401 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57a988-a769-468f-b050-490b7a49ca35 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5b57a988-a769-468f-b050-490b7a49ca35 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:58:39.869433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:39.871072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:39.872302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 625/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:39.876804 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660c38c-a072-4e7f-a8ed-d297f6bad725 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:39.995659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660c38c-a072-4e7f-a8ed-d297f6bad725 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:40.034483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f219468-86ef-4456-ad63-3f38f30fb61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:40.035148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f219468-86ef-4456-ad63-3f38f30fb61f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:40.035425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f219468-86ef-4456-ad63-3f38f30fb61f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:40.035693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f219468-86ef-4456-ad63-3f38f30fb61f None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:40.036425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f219468-86ef-4456-ad63-3f38f30fb61f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:40.036733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f219468-86ef-4456-ad63-3f38f30fb61f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:40.038408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f219468-86ef-4456-ad63-3f38f30fb61f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:40.048399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:40.059214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 643/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Aug 19 20:58:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:58:40.066129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57a988-a769-468f-b050-490b7a49ca35 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:40.178694 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57a988-a769-468f-b050-490b7a49ca35 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:40.212556 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f83465-ee06-468e-9666-71e135c77541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:40.213179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f83465-ee06-468e-9666-71e135c77541 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:40.213478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f83465-ee06-468e-9666-71e135c77541 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:40.213736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f83465-ee06-468e-9666-71e135c77541 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:40.224397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f83465-ee06-468e-9666-71e135c77541 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:40.225884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f83465-ee06-468e-9666-71e135c77541 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-01f83465-ee06-468e-9666-71e135c77541 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:40.239958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:40.241124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:40.242141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:40.242141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:40.242141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:40.242141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 626/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:40.249950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f219468-86ef-4456-ad63-3f38f30fb61f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:40.350191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f219468-86ef-4456-ad63-3f38f30fb61f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:40.384613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e988e-10e4-4621-9fa4-f3caa8e1bba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:40.385218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e988e-10e4-4621-9fa4-f3caa8e1bba6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:40.385502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e988e-10e4-4621-9fa4-f3caa8e1bba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:40.385797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e988e-10e4-4621-9fa4-f3caa8e1bba6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:40.386445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e988e-10e4-4621-9fa4-f3caa8e1bba6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:40.386747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e988e-10e4-4621-9fa4-f3caa8e1bba6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:40.388123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e988e-10e4-4621-9fa4-f3caa8e1bba6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:40.400865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 644/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:40.406699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f83465-ee06-468e-9666-71e135c77541 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:40.438413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:40.513081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f83465-ee06-468e-9666-71e135c77541 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:40.547962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03dd22b-0f5a-422c-b25e-5221e10c2cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:40.548589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dd22b-0f5a-422c-b25e-5221e10c2cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:40.548871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dd22b-0f5a-422c-b25e-5221e10c2cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:40.549146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03dd22b-0f5a-422c-b25e-5221e10c2cf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:40.556448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03dd22b-0f5a-422c-b25e-5221e10c2cf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:40.557923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03dd22b-0f5a-422c-b25e-5221e10c2cf6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d03dd22b-0f5a-422c-b25e-5221e10c2cf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:58:40.569351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:40.570427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:40.571383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:58:40.571383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:58:40.571383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:58:40.571383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:58:40.571383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:58:40.571383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:58:40.571383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:40.571383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:40.571383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 627/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:40.576586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e988e-10e4-4621-9fa4-f3caa8e1bba6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:40.643772 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:40.717195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e988e-10e4-4621-9fa4-f3caa8e1bba6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:40.778760 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91f311-9f82-460e-b57f-5d72e239e9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:40.779373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91f311-9f82-460e-b57f-5d72e239e9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:40.779649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91f311-9f82-460e-b57f-5d72e239e9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:40.779922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91f311-9f82-460e-b57f-5d72e239e9c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:40.780621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91f311-9f82-460e-b57f-5d72e239e9c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:40.780926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91f311-9f82-460e-b57f-5d72e239e9c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:40.782398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91f311-9f82-460e-b57f-5d72e239e9c9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:40.795895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 645/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:58:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:40.802087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03dd22b-0f5a-422c-b25e-5221e10c2cf6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:40.912403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:40.924670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03dd22b-0f5a-422c-b25e-5221e10c2cf6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:40.959665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e228a-5d41-46dd-a1e1-6368cfaa49ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:40.960320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e228a-5d41-46dd-a1e1-6368cfaa49ce None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:40.960600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e228a-5d41-46dd-a1e1-6368cfaa49ce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:40.960869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e228a-5d41-46dd-a1e1-6368cfaa49ce None admin] PATH_INFO: `/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/824165e55d2f4742aa435f1edff9b403/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:40.963270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e228a-5d41-46dd-a1e1-6368cfaa49ce None admin] RBAC: Authorizing `identity:create_grant(project_id=052c34fcd80c4f0e934fb5cdc46cc60b, user_id=824165e55d2f4742aa435f1edff9b403, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:40.964884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e228a-5d41-46dd-a1e1-6368cfaa49ce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:40.980880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 628/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Fri Aug 19 20:58:40 2022] PUT /identity/v3/projects/052c34fcd80c4f0e934fb5cdc46cc60b/users/824165e55d2f4742aa435f1edff9b403/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 20:58:42.906498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f480630-d235-4707-8665-ea688d4454e2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:42.907078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f480630-d235-4707-8665-ea688d4454e2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:42.907078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f480630-d235-4707-8665-ea688d4454e2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:42.908404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 646/1274] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:58:42.913610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba530e-b4b3-4110-9176-b16911a16da8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:42.914155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba530e-b4b3-4110-9176-b16911a16da8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:42.914268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebba530e-b4b3-4110-9176-b16911a16da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:42.933268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ebba530e-b4b3-4110-9176-b16911a16da8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:42.934505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ebba530e-b4b3-4110-9176-b16911a16da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:43.002341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 629/1275] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:58:43.275308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f480630-d235-4707-8665-ea688d4454e2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:43.379554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f480630-d235-4707-8665-ea688d4454e2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:43.398793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:43.416384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e54fe4-e514-474a-a7d8-6602be9377c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:43.417017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e54fe4-e514-474a-a7d8-6602be9377c5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:43.417297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e54fe4-e514-474a-a7d8-6602be9377c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:43.417569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e54fe4-e514-474a-a7d8-6602be9377c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:43.427724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e54fe4-e514-474a-a7d8-6602be9377c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f4e54fe4-e514-474a-a7d8-6602be9377c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 19 20:58:43.433000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:43.434004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:43.434834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 647/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Fri Aug 19 20:58:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:43.442179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba530e-b4b3-4110-9176-b16911a16da8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:43.536577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:43.555722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebba530e-b4b3-4110-9176-b16911a16da8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:43.591312 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958534da-ad7c-462d-8206-34fffef08d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:43.591922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958534da-ad7c-462d-8206-34fffef08d12 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:43.592197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958534da-ad7c-462d-8206-34fffef08d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:43.592468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958534da-ad7c-462d-8206-34fffef08d12 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:43.593134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958534da-ad7c-462d-8206-34fffef08d12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:43.593439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958534da-ad7c-462d-8206-34fffef08d12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:43.594852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958534da-ad7c-462d-8206-34fffef08d12 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:43.604232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 630/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Fri Aug 19 20:58:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:43.610030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e54fe4-e514-474a-a7d8-6602be9377c5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:43.712929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e54fe4-e514-474a-a7d8-6602be9377c5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:43.752848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86467650-d621-4c82-8981-034a2a43792a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:43.753481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86467650-d621-4c82-8981-034a2a43792a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:43.754304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86467650-d621-4c82-8981-034a2a43792a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:43.754304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86467650-d621-4c82-8981-034a2a43792a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:43.765525 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86467650-d621-4c82-8981-034a2a43792a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:43.767015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86467650-d621-4c82-8981-034a2a43792a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-86467650-d621-4c82-8981-034a2a43792a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:43.781673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:43.782937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:43.784022 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:43.784022 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:43.784022 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:43.784022 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 648/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:43.789004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958534da-ad7c-462d-8206-34fffef08d12 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:43.906755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958534da-ad7c-462d-8206-34fffef08d12 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:43.942361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd25ac-e62b-4d69-930b-45db1b6647ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:43.943053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd25ac-e62b-4d69-930b-45db1b6647ee None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:43.943233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd25ac-e62b-4d69-930b-45db1b6647ee None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:43.943505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bd25ac-e62b-4d69-930b-45db1b6647ee None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:43.944144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd25ac-e62b-4d69-930b-45db1b6647ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:43.944448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd25ac-e62b-4d69-930b-45db1b6647ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:43.945832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bd25ac-e62b-4d69-930b-45db1b6647ee None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:43.958908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 631/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:43.965337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86467650-d621-4c82-8981-034a2a43792a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:43.997619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:44.069098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86467650-d621-4c82-8981-034a2a43792a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:44.105010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc42fb9-133c-4e04-b508-748bc5ed377c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:44.105642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc42fb9-133c-4e04-b508-748bc5ed377c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:44.105955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc42fb9-133c-4e04-b508-748bc5ed377c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:44.106233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc42fb9-133c-4e04-b508-748bc5ed377c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:44.113680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc42fb9-133c-4e04-b508-748bc5ed377c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:44.115233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc42fb9-133c-4e04-b508-748bc5ed377c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5cc42fb9-133c-4e04-b508-748bc5ed377c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 19 20:58:44.127223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:44.129444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:44.131644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 649/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:44.134923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd25ac-e62b-4d69-930b-45db1b6647ee None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:44.138756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:44.233576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bd25ac-e62b-4d69-930b-45db1b6647ee None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:44.268322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943279df-3a95-493f-8e45-0e3555a7b7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:44.268922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943279df-3a95-493f-8e45-0e3555a7b7ea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:44.269199 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943279df-3a95-493f-8e45-0e3555a7b7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:44.269474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943279df-3a95-493f-8e45-0e3555a7b7ea None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:44.270235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943279df-3a95-493f-8e45-0e3555a7b7ea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:44.270542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943279df-3a95-493f-8e45-0e3555a7b7ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:44.271982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943279df-3a95-493f-8e45-0e3555a7b7ea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:44.288965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 632/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Fri Aug 19 20:58:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:44.295723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc42fb9-133c-4e04-b508-748bc5ed377c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:44.387738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:44.397997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc42fb9-133c-4e04-b508-748bc5ed377c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:44.439745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cf4bd-68b9-44d3-b8a3-566e31c959cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:44.440341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cf4bd-68b9-44d3-b8a3-566e31c959cb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:44.440615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cf4bd-68b9-44d3-b8a3-566e31c959cb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:44.440882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cf4bd-68b9-44d3-b8a3-566e31c959cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:44.522071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cf4bd-68b9-44d3-b8a3-566e31c959cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:44.523544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cf4bd-68b9-44d3-b8a3-566e31c959cb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:44.616423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:44.629530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 650/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Fri Aug 19 20:58:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 19 20:58:44.635890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943279df-3a95-493f-8e45-0e3555a7b7ea None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:44.700475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:44.740168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943279df-3a95-493f-8e45-0e3555a7b7ea None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:44.776919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c22e6-e937-4992-a1a8-9bf620a0b582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:44.777552 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c22e6-e937-4992-a1a8-9bf620a0b582 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:44.777861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c22e6-e937-4992-a1a8-9bf620a0b582 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:44.778139 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c22e6-e937-4992-a1a8-9bf620a0b582 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:44.788508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c22e6-e937-4992-a1a8-9bf620a0b582 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:44.790089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c22e6-e937-4992-a1a8-9bf620a0b582 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-212c22e6-e937-4992-a1a8-9bf620a0b582 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 19 20:58:44.805070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:44.806933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 19 20:58:44.808180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 19 20:58:44.808180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 19 20:58:44.808180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:44.808180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 633/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 20:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:44.812909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cf4bd-68b9-44d3-b8a3-566e31c959cb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:44.922716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cf4bd-68b9-44d3-b8a3-566e31c959cb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:44.956345 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3fe30-71f8-4950-af66-9e72289d5b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:44.956864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3fe30-71f8-4950-af66-9e72289d5b4f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:44.957145 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3fe30-71f8-4950-af66-9e72289d5b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:44.957419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3fe30-71f8-4950-af66-9e72289d5b4f None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:44.958095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3fe30-71f8-4950-af66-9e72289d5b4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:44.958396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3fe30-71f8-4950-af66-9e72289d5b4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:44.959765 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3fe30-71f8-4950-af66-9e72289d5b4f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:44.971504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 651/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Fri Aug 19 20:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:44.977187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c22e6-e937-4992-a1a8-9bf620a0b582 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:45.078659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c22e6-e937-4992-a1a8-9bf620a0b582 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:45.093719 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:45.115312 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8424315-7afd-463a-9cac-7510f0e6c676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:45.115947 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8424315-7afd-463a-9cac-7510f0e6c676 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:45.116222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8424315-7afd-463a-9cac-7510f0e6c676 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:45.116493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8424315-7afd-463a-9cac-7510f0e6c676 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:45.124497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8424315-7afd-463a-9cac-7510f0e6c676 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:45.125980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8424315-7afd-463a-9cac-7510f0e6c676 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e8424315-7afd-463a-9cac-7510f0e6c676 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 19 20:58:45.137734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:45.138889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:45.140083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 19 20:58:45.140083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 19 20:58:45.140083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 19 20:58:45.140083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 19 20:58:45.140083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 19 20:58:45.140083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 19 20:58:45.140083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 19 20:58:45.140083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Aug 19 20:58:45.140083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 634/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Fri Aug 19 20:58:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:45.144937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3fe30-71f8-4950-af66-9e72289d5b4f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:45.195929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:45.250976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3fe30-71f8-4950-af66-9e72289d5b4f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:45.285349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb98ad3-9143-4e53-8c16-4bdce4a5137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:45.285982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb98ad3-9143-4e53-8c16-4bdce4a5137c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:45.286262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb98ad3-9143-4e53-8c16-4bdce4a5137c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:45.286537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb98ad3-9143-4e53-8c16-4bdce4a5137c None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:45.287237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb98ad3-9143-4e53-8c16-4bdce4a5137c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:45.287536 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb98ad3-9143-4e53-8c16-4bdce4a5137c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:45.288958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb98ad3-9143-4e53-8c16-4bdce4a5137c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:45.301159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 652/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Fri Aug 19 20:58:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:45.308146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8424315-7afd-463a-9cac-7510f0e6c676 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:45.410876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8424315-7afd-463a-9cac-7510f0e6c676 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:45.444757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970c8ea-3625-4e3b-9232-f10cda55a481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:45.445359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970c8ea-3625-4e3b-9232-f10cda55a481 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:45.445636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970c8ea-3625-4e3b-9232-f10cda55a481 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:45.445940 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970c8ea-3625-4e3b-9232-f10cda55a481 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:45.447274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970c8ea-3625-4e3b-9232-f10cda55a481 None admin] RBAC: Adding query filter params (role.id=8b350845f331423cb0494e3449b58310, scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b, user.id=824165e55d2f4742aa435f1edff9b403) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:45.447617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970c8ea-3625-4e3b-9232-f10cda55a481 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:45.449051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970c8ea-3625-4e3b-9232-f10cda55a481 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:45.454841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 635/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Fri Aug 19 20:58:45 2022] GET /identity/v3/role_assignments?user.id=824165e55d2f4742aa435f1edff9b403&scope.project.id=052c34fcd80c4f0e934fb5cdc46cc60b&role.id=8b350845f331423cb0494e3449b58310 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:47.415259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff346f0-219a-4e0f-bff0-9601e71ff9bc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:47.415865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff346f0-219a-4e0f-bff0-9601e71ff9bc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:47.415865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff346f0-219a-4e0f-bff0-9601e71ff9bc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:47.417108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 653/1288] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:58:47.422975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46d45-4596-467f-971f-7b5a603a76fd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:47.423487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46d45-4596-467f-971f-7b5a603a76fd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:47.423587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f46d45-4596-467f-971f-7b5a603a76fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:47.442324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-35f46d45-4596-467f-971f-7b5a603a76fd None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:47.443563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-35f46d45-4596-467f-971f-7b5a603a76fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:47.480480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:47.521291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 636/1289] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:58:47.793616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff346f0-219a-4e0f-bff0-9601e71ff9bc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:47.929705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff346f0-219a-4e0f-bff0-9601e71ff9bc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:47.942554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:47.968703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f1ab9-338f-42c4-8ff2-1e988d1ba116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:47.969336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f1ab9-338f-42c4-8ff2-1e988d1ba116 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:47.969611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f1ab9-338f-42c4-8ff2-1e988d1ba116 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:47.969927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f1ab9-338f-42c4-8ff2-1e988d1ba116 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:47.970514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f1ab9-338f-42c4-8ff2-1e988d1ba116 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:47.971885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9f1ab9-338f-42c4-8ff2-1e988d1ba116 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ae9f1ab9-338f-42c4-8ff2-1e988d1ba116 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:47.981343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:47.982668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:47.983732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 654/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Aug 19 20:58:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:47.988820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f46d45-4596-467f-971f-7b5a603a76fd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:48.094138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f46d45-4596-467f-971f-7b5a603a76fd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:48.129071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a632f-0808-401a-b408-838cf6963618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:48.129680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a632f-0808-401a-b408-838cf6963618 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:48.129991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a632f-0808-401a-b408-838cf6963618 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:48.130269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a632f-0808-401a-b408-838cf6963618 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:48.130907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a632f-0808-401a-b408-838cf6963618 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:48.131214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a632f-0808-401a-b408-838cf6963618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:48.132587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a632f-0808-401a-b408-838cf6963618 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:48.137739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 637/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Aug 19 20:58:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:48.143632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f1ab9-338f-42c4-8ff2-1e988d1ba116 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:48.246932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f1ab9-338f-42c4-8ff2-1e988d1ba116 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:48.286509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca656d-5ba7-4fdd-89f7-406e4b234da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:48.287140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca656d-5ba7-4fdd-89f7-406e4b234da9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:48.287420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca656d-5ba7-4fdd-89f7-406e4b234da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:48.287693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca656d-5ba7-4fdd-89f7-406e4b234da9 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:48.288341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca656d-5ba7-4fdd-89f7-406e4b234da9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:48.288647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca656d-5ba7-4fdd-89f7-406e4b234da9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:48.290659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca656d-5ba7-4fdd-89f7-406e4b234da9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:48.295836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 655/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Aug 19 20:58:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:50.298985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c927a-e8c0-4b46-a3cc-61aaed1b151e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:50.299553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c927a-e8c0-4b46-a3cc-61aaed1b151e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:50.299553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c927a-e8c0-4b46-a3cc-61aaed1b151e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:50.300855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 638/1293] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:58:50.306834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cded4-5e29-4d5a-92ce-86a2056da713 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:50.307581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cded4-5e29-4d5a-92ce-86a2056da713 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:50.307735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9cded4-5e29-4d5a-92ce-86a2056da713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:50.327660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1c9cded4-5e29-4d5a-92ce-86a2056da713 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:50.328889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1c9cded4-5e29-4d5a-92ce-86a2056da713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:50.361806 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:50.408249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 656/1294] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 19 20:58:50.689816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c927a-e8c0-4b46-a3cc-61aaed1b151e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:50.722729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:50.807234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c927a-e8c0-4b46-a3cc-61aaed1b151e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:50.855169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa72d36-264f-47b5-8847-ec5bf37a478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:50.856007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa72d36-264f-47b5-8847-ec5bf37a478b None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:50.856546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa72d36-264f-47b5-8847-ec5bf37a478b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:50.857063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa72d36-264f-47b5-8847-ec5bf37a478b None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:50.858559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa72d36-264f-47b5-8847-ec5bf37a478b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:50.860772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa72d36-264f-47b5-8847-ec5bf37a478b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:50.882176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 639/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Fri Aug 19 20:58:50 2022] POST /identity/v3/services => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 20:58:52.891070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9b764-7644-4a34-a3ea-2c1502810b79 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:52.891656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9b764-7644-4a34-a3ea-2c1502810b79 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:52.891656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9b764-7644-4a34-a3ea-2c1502810b79 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:52.892988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 657/1296] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 20:58:52.898746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2ad30-7bce-4f76-a5e9-0179baea6ae1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:52.899818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2ad30-7bce-4f76-a5e9-0179baea6ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:52.899818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2ad30-7bce-4f76-a5e9-0179baea6ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:52.919190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-83f2ad30-7bce-4f76-a5e9-0179baea6ae1 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:52.920422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83f2ad30-7bce-4f76-a5e9-0179baea6ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:52.999811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 640/1297] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:58:53.278088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9b764-7644-4a34-a3ea-2c1502810b79 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:53.385749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9b764-7644-4a34-a3ea-2c1502810b79 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:53.422339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c83a9-d64d-43c3-94b0-e8daf5f388a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:53.422968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c83a9-d64d-43c3-94b0-e8daf5f388a7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:53.423245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c83a9-d64d-43c3-94b0-e8daf5f388a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:53.423516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c83a9-d64d-43c3-94b0-e8daf5f388a7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:53.424101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c83a9-d64d-43c3-94b0-e8daf5f388a7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:53.425475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8c83a9-d64d-43c3-94b0-e8daf5f388a7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7b8c83a9-d64d-43c3-94b0-e8daf5f388a7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:53.434668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:53.436576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:53.438255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 658/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Aug 19 20:58:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:53.446479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2ad30-7bce-4f76-a5e9-0179baea6ae1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:53.479860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:53.628519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2ad30-7bce-4f76-a5e9-0179baea6ae1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:53.690374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e640c-c271-4743-9448-b05c4c663da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:53.691381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e640c-c271-4743-9448-b05c4c663da7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:53.691863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e640c-c271-4743-9448-b05c4c663da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:53.692336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e640c-c271-4743-9448-b05c4c663da7 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:53.693439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e640c-c271-4743-9448-b05c4c663da7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:53.694000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e640c-c271-4743-9448-b05c4c663da7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:53.697187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e640c-c271-4743-9448-b05c4c663da7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:53.706781 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 641/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Aug 19 20:58:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:53.713273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c83a9-d64d-43c3-94b0-e8daf5f388a7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:53.748642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:53.825014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c83a9-d64d-43c3-94b0-e8daf5f388a7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:53.876183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c3c4c-adcb-47dc-a618-6e2096204728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:53.876830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c3c4c-adcb-47dc-a618-6e2096204728 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:53.877111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c3c4c-adcb-47dc-a618-6e2096204728 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:53.877387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c3c4c-adcb-47dc-a618-6e2096204728 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:53.878218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c3c4c-adcb-47dc-a618-6e2096204728 None admin] RBAC: Adding query filter params (interface=public, service_id=ecc83f4d8be04038b7375886307f0524, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:53.878597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c3c4c-adcb-47dc-a618-6e2096204728 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:53.880086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c3c4c-adcb-47dc-a618-6e2096204728 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:53.891920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 659/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Fri Aug 19 20:58:53 2022] GET /identity/v3/endpoints?service_id=ecc83f4d8be04038b7375886307f0524&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 20:58:53.898094 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e640c-c271-4743-9448-b05c4c663da7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:54.013149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:54.033713 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e640c-c271-4743-9448-b05c4c663da7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:54.069149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2281a9-3f0c-4ba8-ab7c-40ee1976370b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:54.069784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2281a9-3f0c-4ba8-ab7c-40ee1976370b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:54.070075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2281a9-3f0c-4ba8-ab7c-40ee1976370b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:54.070352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2281a9-3f0c-4ba8-ab7c-40ee1976370b None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:54.071515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2281a9-3f0c-4ba8-ab7c-40ee1976370b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:54.072914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2281a9-3f0c-4ba8-ab7c-40ee1976370b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:54.080050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 642/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Fri Aug 19 20:58:53 2022] GET /identity/v3/services => generated 2191 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 20:58:56.034377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb8a3a-31fd-4013-b3f7-72299e01c055 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:56.034982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb8a3a-31fd-4013-b3f7-72299e01c055 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:56.034982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb8a3a-31fd-4013-b3f7-72299e01c055 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:56.036310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 660/1302] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 20:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 20:58:56.042175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69f51c-e2a8-479e-ba08-185bff04e2ac None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:56.042706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69f51c-e2a8-479e-ba08-185bff04e2ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:56.042706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d69f51c-e2a8-479e-ba08-185bff04e2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:56.062703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4d69f51c-e2a8-479e-ba08-185bff04e2ac None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 20:58:56.063919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d69f51c-e2a8-479e-ba08-185bff04e2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 20:58:56.150855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 643/1303] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 20:58:56 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 19 20:58:56.444739 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb8a3a-31fd-4013-b3f7-72299e01c055 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:56.556431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:56.563895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb8a3a-31fd-4013-b3f7-72299e01c055 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:56.607158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878d178-eea7-44f6-959b-0e35b53a89ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:56.607866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878d178-eea7-44f6-959b-0e35b53a89ff None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:56.608191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878d178-eea7-44f6-959b-0e35b53a89ff None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:56.608652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1878d178-eea7-44f6-959b-0e35b53a89ff None admin] PATH_INFO: `/v3/services/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:56.609297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878d178-eea7-44f6-959b-0e35b53a89ff None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:56.611058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1878d178-eea7-44f6-959b-0e35b53a89ff None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1878d178-eea7-44f6-959b-0e35b53a89ff None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 19 20:58:56.628447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 19 20:58:56.631596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Aug 19 20:58:56.633437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 661/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Fri Aug 19 20:58:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 19 20:58:56.640294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69f51c-e2a8-479e-ba08-185bff04e2ac None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:56.743325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:58:56.771365 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d69f51c-e2a8-479e-ba08-185bff04e2ac None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:56.811686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20ddf1-d906-44fe-a0b7-1bc2cf82a5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:56.812304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20ddf1-d906-44fe-a0b7-1bc2cf82a5da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:56.812579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20ddf1-d906-44fe-a0b7-1bc2cf82a5da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:56.812849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20ddf1-d906-44fe-a0b7-1bc2cf82a5da None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:56.813520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20ddf1-d906-44fe-a0b7-1bc2cf82a5da None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 20:58:56.813844 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20ddf1-d906-44fe-a0b7-1bc2cf82a5da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:56.815232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20ddf1-d906-44fe-a0b7-1bc2cf82a5da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:56.822148 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 644/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Fri Aug 19 20:58:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 20:58:56.828310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878d178-eea7-44f6-959b-0e35b53a89ff None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 20:58:56.951334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1878d178-eea7-44f6-959b-0e35b53a89ff None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 20:58:56.994064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dde5f2-aa88-4782-98d5-19d96a3e0e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 20:58:56.994764 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dde5f2-aa88-4782-98d5-19d96a3e0e00 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 20:58:56.995255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dde5f2-aa88-4782-98d5-19d96a3e0e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 20:58:56.996019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dde5f2-aa88-4782-98d5-19d96a3e0e00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 20:58:56.997241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dde5f2-aa88-4782-98d5-19d96a3e0e00 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 20:58:56.999159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dde5f2-aa88-4782-98d5-19d96a3e0e00 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 20:58:57.022061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 662/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 20:58:56 2022] POST /identity/v3/endpoints => generated 331 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 20:59:06.817806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=30 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 20:59:07.017583 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:04.980221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214724a7-b29e-481e-bcf5-a77a13cec4f9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:04.980830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214724a7-b29e-481e-bcf5-a77a13cec4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:04.980830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214724a7-b29e-481e-bcf5-a77a13cec4f9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:04.988138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3861b09-88c1-4bf8-93cb-6ce1993da27f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:04.988708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3861b09-88c1-4bf8-93cb-6ce1993da27f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:04.988708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3861b09-88c1-4bf8-93cb-6ce1993da27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:04.994568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 645/1307] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:04 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:05.138818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c3861b09-88c1-4bf8-93cb-6ce1993da27f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:05.141088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c3861b09-88c1-4bf8-93cb-6ce1993da27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:05.422571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 663/1308] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:00:05.756430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b5728-9fd0-485c-bc21-6e4bc0e80a05 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:05.757069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b5728-9fd0-485c-bc21-6e4bc0e80a05 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:05.757752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b5728-9fd0-485c-bc21-6e4bc0e80a05 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:05.760693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 646/1309] 10.222.0.16 () {54 vars in 965 bytes} [Fri Aug 19 21:00:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:05.774499 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086e460-4c54-4efb-a5de-7194f9f214fc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:05.774849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086e460-4c54-4efb-a5de-7194f9f214fc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:05.775055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086e460-4c54-4efb-a5de-7194f9f214fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:05.872079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3086e460-4c54-4efb-a5de-7194f9f214fc None None] MFA Rules not processed for user `b37a4275e7c6441aa9e8bb0298f56589`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:05.873233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3086e460-4c54-4efb-a5de-7194f9f214fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:05.896408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:05.963049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 664/1310] 10.222.0.16 () {58 vars in 1062 bytes} [Fri Aug 19 21:00:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:00:05.979090 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b5728-9fd0-485c-bc21-6e4bc0e80a05 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:05.981163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-014b5728-9fd0-485c-bc21-6e4bc0e80a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:06.201969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b5728-9fd0-485c-bc21-6e4bc0e80a05 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:06.370516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73029c9-31f4-4c2c-8438-9c35bf4ad4c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:06.372017 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73029c9-31f4-4c2c-8438-9c35bf4ad4c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:06.372512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73029c9-31f4-4c2c-8438-9c35bf4ad4c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:06.373002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73029c9-31f4-4c2c-8438-9c35bf4ad4c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:06.376205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c73029c9-31f4-4c2c-8438-9c35bf4ad4c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:06.500829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:06.632886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73029c9-31f4-4c2c-8438-9c35bf4ad4c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:06.636205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73029c9-31f4-4c2c-8438-9c35bf4ad4c3 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:06.696030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 647/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 717 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:00:09.173090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd2936-ddb1-4869-8948-33e6ce1a0bcb None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:09.173651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd2936-ddb1-4869-8948-33e6ce1a0bcb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:09.173651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd2936-ddb1-4869-8948-33e6ce1a0bcb None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:09.175032 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 665/1312] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:09.181604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51544a0-a708-4281-9186-a5e641258d0c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:09.182142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51544a0-a708-4281-9186-a5e641258d0c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:09.182267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51544a0-a708-4281-9186-a5e641258d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:09.271751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e51544a0-a708-4281-9186-a5e641258d0c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:09.273829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e51544a0-a708-4281-9186-a5e641258d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:09.387977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 648/1313] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:00:09.685687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd2936-ddb1-4869-8948-33e6ce1a0bcb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:09.687251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04dd2936-ddb1-4869-8948-33e6ce1a0bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:09.811225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd2936-ddb1-4869-8948-33e6ce1a0bcb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:09.887647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:09.890126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dcb16b-7eae-4dab-addf-17587f37907c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:09.890747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcb16b-7eae-4dab-addf-17587f37907c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:09.891031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcb16b-7eae-4dab-addf-17587f37907c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:09.891506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dcb16b-7eae-4dab-addf-17587f37907c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:09.893680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11dcb16b-7eae-4dab-addf-17587f37907c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:10.011302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcb16b-7eae-4dab-addf-17587f37907c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:10.012773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dcb16b-7eae-4dab-addf-17587f37907c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:10.036449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 666/1314] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:00:12.344785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c125d24-5b08-4660-b0e6-68d789c0f244 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:12.345345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c125d24-5b08-4660-b0e6-68d789c0f244 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:12.345345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c125d24-5b08-4660-b0e6-68d789c0f244 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:12.346652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 649/1315] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:12.352691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb22cb-fa6e-45cc-a196-ab203ef45bb2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:12.353190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb22cb-fa6e-45cc-a196-ab203ef45bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:12.353326 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb22cb-fa6e-45cc-a196-ab203ef45bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:12.433601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8ddb22cb-fa6e-45cc-a196-ab203ef45bb2 None None] MFA Rules not processed for user `f11433fdb39c41a9b2b8dd15ec05cb21`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:12.434720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8ddb22cb-fa6e-45cc-a196-ab203ef45bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:12.495889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:12.562948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 667/1316] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:00:12.850267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c125d24-5b08-4660-b0e6-68d789c0f244 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:12.851865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5c125d24-5b08-4660-b0e6-68d789c0f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:12.862083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:12.977276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c125d24-5b08-4660-b0e6-68d789c0f244 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:13.064166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a771f6a-9ae2-4a91-99dc-6ba5fb495c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:13.064774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a771f6a-9ae2-4a91-99dc-6ba5fb495c73 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:13.065047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a771f6a-9ae2-4a91-99dc-6ba5fb495c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:13.065312 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a771f6a-9ae2-4a91-99dc-6ba5fb495c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:13.067009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8a771f6a-9ae2-4a91-99dc-6ba5fb495c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:13.136805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:13.187183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a771f6a-9ae2-4a91-99dc-6ba5fb495c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:13.188662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a771f6a-9ae2-4a91-99dc-6ba5fb495c73 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:13.208639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 650/1317] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:00:15.827122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c8b08-56de-42a0-a43d-e355bc6664f4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:15.827680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c8b08-56de-42a0-a43d-e355bc6664f4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:15.827680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c8b08-56de-42a0-a43d-e355bc6664f4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:15.829417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 668/1318] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:15.835979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a4d5-8f59-442c-9f48-71c856527ac2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:15.837021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a4d5-8f59-442c-9f48-71c856527ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:15.837021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4a4d5-8f59-442c-9f48-71c856527ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:15.916698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e2f4a4d5-8f59-442c-9f48-71c856527ac2 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:15.918068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e2f4a4d5-8f59-442c-9f48-71c856527ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:16.055143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 651/1319] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:00:16.454999 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c8b08-56de-42a0-a43d-e355bc6664f4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:16.456542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-020c8b08-56de-42a0-a43d-e355bc6664f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:16.575898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c8b08-56de-42a0-a43d-e355bc6664f4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:16.670152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda4045-2241-45f5-b575-9b84a9197e9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:16.670870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda4045-2241-45f5-b575-9b84a9197e9e service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:16.671203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda4045-2241-45f5-b575-9b84a9197e9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:16.671566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda4045-2241-45f5-b575-9b84a9197e9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:16.673488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5fda4045-2241-45f5-b575-9b84a9197e9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:16.683135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:16.838024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda4045-2241-45f5-b575-9b84a9197e9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:16.840419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fda4045-2241-45f5-b575-9b84a9197e9e service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:16.867333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 669/1320] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:00:19.757362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5de6ec-807c-4cdd-b3ec-33a123f14e6f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:19.757974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5de6ec-807c-4cdd-b3ec-33a123f14e6f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:19.758077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5de6ec-807c-4cdd-b3ec-33a123f14e6f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:19.759508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 652/1321] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:19.765463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8beba2-397a-4ad0-a6cc-fb1d09b26da4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:19.766100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8beba2-397a-4ad0-a6cc-fb1d09b26da4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:19.766100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8beba2-397a-4ad0-a6cc-fb1d09b26da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:19.843537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0f8beba2-397a-4ad0-a6cc-fb1d09b26da4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:19.844797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0f8beba2-397a-4ad0-a6cc-fb1d09b26da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:19.917642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:19.966917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 670/1322] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:00:20.271324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5de6ec-807c-4cdd-b3ec-33a123f14e6f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:20.274656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0e5de6ec-807c-4cdd-b3ec-33a123f14e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:20.312194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:20.445628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5de6ec-807c-4cdd-b3ec-33a123f14e6f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:20.544784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d439e57-6bba-4978-87b2-bb8c4e6287aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:20.545456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d439e57-6bba-4978-87b2-bb8c4e6287aa service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:20.545852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d439e57-6bba-4978-87b2-bb8c4e6287aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:20.546189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d439e57-6bba-4978-87b2-bb8c4e6287aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:20.548144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1d439e57-6bba-4978-87b2-bb8c4e6287aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:20.638077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:20.682876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d439e57-6bba-4978-87b2-bb8c4e6287aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:20.684344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d439e57-6bba-4978-87b2-bb8c4e6287aa service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:20.705282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 653/1323] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:00:22.735891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c491a7-b961-4fb8-a6fe-a5b82d9a0e5e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:22.736430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c491a7-b961-4fb8-a6fe-a5b82d9a0e5e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:22.736430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c491a7-b961-4fb8-a6fe-a5b82d9a0e5e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:22.737873 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 671/1324] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:22.744380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831cdf4-ac67-4355-b615-d47025f27cb6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:22.744981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831cdf4-ac67-4355-b615-d47025f27cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:22.744981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831cdf4-ac67-4355-b615-d47025f27cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:22.818801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9831cdf4-ac67-4355-b615-d47025f27cb6 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:22.820751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9831cdf4-ac67-4355-b615-d47025f27cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:22.935947 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 654/1325] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:00:23.245902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae177-3476-4f16-a1c6-a89cd936557a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:23.246454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae177-3476-4f16-a1c6-a89cd936557a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:23.246741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fae177-3476-4f16-a1c6-a89cd936557a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:23.248115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 672/1326] 10.222.0.16 () {54 vars in 965 bytes} [Fri Aug 19 21:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:23.254942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfd713-69bb-4ea7-b9e3-05b606b2c86b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:23.255344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfd713-69bb-4ea7-b9e3-05b606b2c86b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:23.255711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bfd713-69bb-4ea7-b9e3-05b606b2c86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:23.293169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:23.348904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-33bfd713-69bb-4ea7-b9e3-05b606b2c86b None None] MFA Rules not processed for user `b37a4275e7c6441aa9e8bb0298f56589`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:23.350282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-33bfd713-69bb-4ea7-b9e3-05b606b2c86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:23.453318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 655/1327] 10.222.0.16 () {58 vars in 1062 bytes} [Fri Aug 19 21:00:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:00:23.463344 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fae177-3476-4f16-a1c6-a89cd936557a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:23.465371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d6fae177-3476-4f16-a1c6-a89cd936557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:23.602048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fae177-3476-4f16-a1c6-a89cd936557a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:23.686215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33051c-dac1-4894-bf25-3070b40d8c09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:23.686839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33051c-dac1-4894-bf25-3070b40d8c09 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:23.687120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33051c-dac1-4894-bf25-3070b40d8c09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:23.687398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33051c-dac1-4894-bf25-3070b40d8c09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:23.689084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4a33051c-dac1-4894-bf25-3070b40d8c09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:23.708167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:23.813128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33051c-dac1-4894-bf25-3070b40d8c09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:23.814780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33051c-dac1-4894-bf25-3070b40d8c09 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:23.841303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 673/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:00:26.218240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554ddfb-ee50-4093-89cb-06c4aaa2a2e5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:26.219077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554ddfb-ee50-4093-89cb-06c4aaa2a2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:26.219077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554ddfb-ee50-4093-89cb-06c4aaa2a2e5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:26.220051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 656/1329] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:26.227860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218662e-c2b2-4bf9-bad8-663d6445a846 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:26.228705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218662e-c2b2-4bf9-bad8-663d6445a846 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:26.228705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6218662e-c2b2-4bf9-bad8-663d6445a846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:26.317868 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6218662e-c2b2-4bf9-bad8-663d6445a846 None None] MFA Rules not processed for user `f11433fdb39c41a9b2b8dd15ec05cb21`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:26.319683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6218662e-c2b2-4bf9-bad8-663d6445a846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:26.391199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:26.451991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 674/1330] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:26 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:00:26.740843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554ddfb-ee50-4093-89cb-06c4aaa2a2e5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:26.742635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9554ddfb-ee50-4093-89cb-06c4aaa2a2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:26.903800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554ddfb-ee50-4093-89cb-06c4aaa2a2e5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:26.928284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:26.994637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3404a639-b29e-4221-8c53-851681af65f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:26.995695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404a639-b29e-4221-8c53-851681af65f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:26.996832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404a639-b29e-4221-8c53-851681af65f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:26.997303 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3404a639-b29e-4221-8c53-851681af65f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:26.999307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3404a639-b29e-4221-8c53-851681af65f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:27.210323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404a639-b29e-4221-8c53-851681af65f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:27.212996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3404a639-b29e-4221-8c53-851681af65f9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:27.247534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 657/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:26 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:00:29.693400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb6d88-e33f-418b-851d-2cfcf40e17b0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:29.694553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb6d88-e33f-418b-851d-2cfcf40e17b0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:29.695270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eb6d88-e33f-418b-851d-2cfcf40e17b0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:29.696680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 675/1332] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:29.702814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37f891-c7ce-46bf-9524-ebf470598ed2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:29.703334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37f891-c7ce-46bf-9524-ebf470598ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:29.703436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37f891-c7ce-46bf-9524-ebf470598ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:29.775336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:29.779392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ce37f891-c7ce-46bf-9524-ebf470598ed2 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:29.781030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce37f891-c7ce-46bf-9524-ebf470598ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:29.888530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 658/1333] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:00:30.175900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb6d88-e33f-418b-851d-2cfcf40e17b0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:30.177624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c9eb6d88-e33f-418b-851d-2cfcf40e17b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:30.306755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eb6d88-e33f-418b-851d-2cfcf40e17b0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:30.400622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebca957-f658-4a23-b2ce-b91ffe8f0337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:30.401152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebca957-f658-4a23-b2ce-b91ffe8f0337 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:30.401440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebca957-f658-4a23-b2ce-b91ffe8f0337 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:30.401715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebca957-f658-4a23-b2ce-b91ffe8f0337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:30.403391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bebca957-f658-4a23-b2ce-b91ffe8f0337 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:30.442307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:30.555758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebca957-f658-4a23-b2ce-b91ffe8f0337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:30.557210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebca957-f658-4a23-b2ce-b91ffe8f0337 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:30.582099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 676/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:00:32.644831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64f7d7-9322-4b37-8ff3-3e256cf724a2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:32.645379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64f7d7-9322-4b37-8ff3-3e256cf724a2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:32.645379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64f7d7-9322-4b37-8ff3-3e256cf724a2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:32.646736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 659/1335] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:32.653081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f1d0-55f9-4e96-911f-24ee036c7dc3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:32.653541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f1d0-55f9-4e96-911f-24ee036c7dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:32.653691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c40f1d0-55f9-4e96-911f-24ee036c7dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:32.745415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6c40f1d0-55f9-4e96-911f-24ee036c7dc3 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:32.747113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6c40f1d0-55f9-4e96-911f-24ee036c7dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:32.854728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:32.889018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 677/1336] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:00:33.214564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64f7d7-9322-4b37-8ff3-3e256cf724a2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:33.216123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-da64f7d7-9322-4b37-8ff3-3e256cf724a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:33.378063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64f7d7-9322-4b37-8ff3-3e256cf724a2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:33.409930 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:33.482121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b83dd4-028a-432d-8cf3-9644778b7744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:33.482749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b83dd4-028a-432d-8cf3-9644778b7744 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:33.483031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b83dd4-028a-432d-8cf3-9644778b7744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:33.483299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b83dd4-028a-432d-8cf3-9644778b7744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:33.484987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-97b83dd4-028a-432d-8cf3-9644778b7744 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:33.634251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b83dd4-028a-432d-8cf3-9644778b7744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:33.635643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b83dd4-028a-432d-8cf3-9644778b7744 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:33.657235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 660/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:00:36.224652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc2f7f-2053-4249-b413-f91b66af5e91 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:36.225253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc2f7f-2053-4249-b413-f91b66af5e91 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:36.225253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fc2f7f-2053-4249-b413-f91b66af5e91 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:36.227163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 678/1338] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:36.233081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34deab-e9bb-4ad5-aba8-a7d19908a5fe None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:36.233592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34deab-e9bb-4ad5-aba8-a7d19908a5fe None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:36.233702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca34deab-e9bb-4ad5-aba8-a7d19908a5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:36.302409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ca34deab-e9bb-4ad5-aba8-a7d19908a5fe None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:36.303676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ca34deab-e9bb-4ad5-aba8-a7d19908a5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:36.311962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:36.409071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 661/1339] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:00:36.695297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc2f7f-2053-4249-b413-f91b66af5e91 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:36.696856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-08fc2f7f-2053-4249-b413-f91b66af5e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:36.819926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fc2f7f-2053-4249-b413-f91b66af5e91 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:36.902715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094606d-bebd-4cae-a3fd-f10682cb4a97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:36.903336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094606d-bebd-4cae-a3fd-f10682cb4a97 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:36.903621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094606d-bebd-4cae-a3fd-f10682cb4a97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:36.903893 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094606d-bebd-4cae-a3fd-f10682cb4a97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:36.905575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a094606d-bebd-4cae-a3fd-f10682cb4a97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:36.944331 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:37.021098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094606d-bebd-4cae-a3fd-f10682cb4a97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:37.023235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094606d-bebd-4cae-a3fd-f10682cb4a97 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:37.043504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 679/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:00:41.322902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9437b-1991-4381-8a0c-6dcf495b1a9a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:41.323442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9437b-1991-4381-8a0c-6dcf495b1a9a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:41.323442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca9437b-1991-4381-8a0c-6dcf495b1a9a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:41.324788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 662/1341] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:41.330855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833427f-3c45-4b4a-8cee-fa100cec0025 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:41.331365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833427f-3c45-4b4a-8cee-fa100cec0025 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:41.331448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833427f-3c45-4b4a-8cee-fa100cec0025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:41.407241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4833427f-3c45-4b4a-8cee-fa100cec0025 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:41.408568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4833427f-3c45-4b4a-8cee-fa100cec0025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:41.497834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:41.510889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 680/1342] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:00:41.806985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9437b-1991-4381-8a0c-6dcf495b1a9a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:41.808641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4ca9437b-1991-4381-8a0c-6dcf495b1a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:41.922057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca9437b-1991-4381-8a0c-6dcf495b1a9a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:41.962887 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:42.002412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6ad17-5b2d-4ce6-bb39-f89ecfc0256e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:42.003024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6ad17-5b2d-4ce6-bb39-f89ecfc0256e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:42.003305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6ad17-5b2d-4ce6-bb39-f89ecfc0256e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:42.003575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6ad17-5b2d-4ce6-bb39-f89ecfc0256e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:42.005191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e0b6ad17-5b2d-4ce6-bb39-f89ecfc0256e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:42.124688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6ad17-5b2d-4ce6-bb39-f89ecfc0256e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:42.126389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6ad17-5b2d-4ce6-bb39-f89ecfc0256e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:42.148196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 663/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:00:44.755973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46862-eb52-49ba-bc0b-43ac90916691 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:44.756980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46862-eb52-49ba-bc0b-43ac90916691 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:44.758057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46862-eb52-49ba-bc0b-43ac90916691 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:44.759961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 681/1344] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:44.764864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fef29-42e0-4b2f-994a-2f7e12abe507 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:44.765424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fef29-42e0-4b2f-994a-2f7e12abe507 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:44.765529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fef29-42e0-4b2f-994a-2f7e12abe507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:44.854071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b72fef29-42e0-4b2f-994a-2f7e12abe507 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:44.855797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b72fef29-42e0-4b2f-994a-2f7e12abe507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:44.878798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:44.977060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 664/1345] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:00:45.269920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa46862-eb52-49ba-bc0b-43ac90916691 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:45.272273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8fa46862-eb52-49ba-bc0b-43ac90916691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:45.382308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa46862-eb52-49ba-bc0b-43ac90916691 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:45.463286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e006230-be72-4cc9-8834-067917c2a43f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:45.464314 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e006230-be72-4cc9-8834-067917c2a43f service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:45.464812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e006230-be72-4cc9-8834-067917c2a43f service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:45.465292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e006230-be72-4cc9-8834-067917c2a43f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:45.467991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0e006230-be72-4cc9-8834-067917c2a43f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:45.525574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:45.608658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e006230-be72-4cc9-8834-067917c2a43f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:45.610166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e006230-be72-4cc9-8834-067917c2a43f service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:45.635643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 682/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:00:49.652266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1424e4-cb27-4163-9fef-964fc9942be4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:49.652840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1424e4-cb27-4163-9fef-964fc9942be4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:49.652840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1424e4-cb27-4163-9fef-964fc9942be4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:49.654257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 665/1347] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:49.660058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cc8d7-482b-43a2-b2b1-326d5082bb5f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:49.661122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cc8d7-482b-43a2-b2b1-326d5082bb5f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:49.661403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041cc8d7-482b-43a2-b2b1-326d5082bb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:49.753713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-041cc8d7-482b-43a2-b2b1-326d5082bb5f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:49.756723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-041cc8d7-482b-43a2-b2b1-326d5082bb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:49.873113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:49.876051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 683/1348] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:00:50.160498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1424e4-cb27-4163-9fef-964fc9942be4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:00:50.162772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec1424e4-cb27-4163-9fef-964fc9942be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:50.278571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1424e4-cb27-4163-9fef-964fc9942be4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:00:50.328390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:50.359303 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38312d63-8270-4751-b051-7cdfc322e8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:00:50.359921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38312d63-8270-4751-b051-7cdfc322e8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:50.360198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38312d63-8270-4751-b051-7cdfc322e8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:50.360472 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38312d63-8270-4751-b051-7cdfc322e8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:50.362158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-38312d63-8270-4751-b051-7cdfc322e8d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:50.480948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38312d63-8270-4751-b051-7cdfc322e8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:00:50.482437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38312d63-8270-4751-b051-7cdfc322e8d8 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:00:50.502608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 666/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:00:55.772216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba9b39-5563-4a74-b3c5-ed731cc28396 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:55.772755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba9b39-5563-4a74-b3c5-ed731cc28396 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:55.772755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba9b39-5563-4a74-b3c5-ed731cc28396 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:55.774672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 684/1350] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:00:55.780332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53163272-8755-4a8c-a58c-529e7b7df4d9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:00:55.781140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53163272-8755-4a8c-a58c-529e7b7df4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:00:55.781602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53163272-8755-4a8c-a58c-529e7b7df4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:00:55.920069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-53163272-8755-4a8c-a58c-529e7b7df4d9 None None] MFA Rules not processed for user `824165e55d2f4742aa435f1edff9b403`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:00:55.921581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-53163272-8755-4a8c-a58c-529e7b7df4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:00:55.947550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:00:56.079407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 667/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:00:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:01:00.463154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd14bb-412c-48f7-bd2f-e333433cac72 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:00.463667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd14bb-412c-48f7-bd2f-e333433cac72 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:00.463667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd14bb-412c-48f7-bd2f-e333433cac72 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:00.465144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 685/1352] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:00.470353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af896b-89f4-46ae-8837-456ec89bbca6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:00.471221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af896b-89f4-46ae-8837-456ec89bbca6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:00.471803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af896b-89f4-46ae-8837-456ec89bbca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:00.567334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-41af896b-89f4-46ae-8837-456ec89bbca6 None None] MFA Rules not processed for user `824165e55d2f4742aa435f1edff9b403`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:00.568957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-41af896b-89f4-46ae-8837-456ec89bbca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:00.651529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:00.716399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 668/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:01:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:02.094790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72a67b-19b8-472a-8b83-fe2ead9a0249 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:02.095301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72a67b-19b8-472a-8b83-fe2ead9a0249 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:02.095397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d72a67b-19b8-472a-8b83-fe2ead9a0249 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:02.096743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 686/1354] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:02.102933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a147b-a23c-4f2c-a3b5-44938d22ee14 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:02.103407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a147b-a23c-4f2c-a3b5-44938d22ee14 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:02.103499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a147b-a23c-4f2c-a3b5-44938d22ee14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:02.185478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-921a147b-a23c-4f2c-a3b5-44938d22ee14 None None] MFA Rules not processed for user `824165e55d2f4742aa435f1edff9b403`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:02.186768 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-921a147b-a23c-4f2c-a3b5-44938d22ee14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:02.297324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 669/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:01:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:05.740413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc06379-7480-4be5-a8d3-eee3d791b681 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:05.741013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc06379-7480-4be5-a8d3-eee3d791b681 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:05.741301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc06379-7480-4be5-a8d3-eee3d791b681 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:05.742753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 687/1356] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:05.748566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a967851-87f3-49c7-b3d2-01fa45aafd0e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:05.749031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a967851-87f3-49c7-b3d2-01fa45aafd0e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:05.749146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a967851-87f3-49c7-b3d2-01fa45aafd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:05.793970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:05.828733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3a967851-87f3-49c7-b3d2-01fa45aafd0e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:05.830079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a967851-87f3-49c7-b3d2-01fa45aafd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:05.963659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 670/1357] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:06.341815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f835b18-fce7-435f-bd72-66ad402e60c2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:06.342360 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f835b18-fce7-435f-bd72-66ad402e60c2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:06.342659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f835b18-fce7-435f-bd72-66ad402e60c2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:06.344467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 688/1358] 10.222.0.16 () {54 vars in 962 bytes} [Fri Aug 19 21:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:06.356024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeac200-498c-4930-b9b3-2604241bb30e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:06.356926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeac200-498c-4930-b9b3-2604241bb30e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:06.357640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeac200-498c-4930-b9b3-2604241bb30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:06.460581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-edeac200-498c-4930-b9b3-2604241bb30e None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:06.461881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-edeac200-498c-4930-b9b3-2604241bb30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:06.518484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:06.604648 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 671/1359] 10.222.0.16 () {58 vars in 1059 bytes} [Fri Aug 19 21:01:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:06.613239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f835b18-fce7-435f-bd72-66ad402e60c2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:06.615463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1f835b18-fce7-435f-bd72-66ad402e60c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:06.784797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f835b18-fce7-435f-bd72-66ad402e60c2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:06.868285 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:06.915597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb101e4-b418-4280-8295-56a4bdf24f17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:06.916392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb101e4-b418-4280-8295-56a4bdf24f17 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:06.916807 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb101e4-b418-4280-8295-56a4bdf24f17 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:06.917160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb101e4-b418-4280-8295-56a4bdf24f17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:06.918938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5bb101e4-b418-4280-8295-56a4bdf24f17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:07.016973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb101e4-b418-4280-8295-56a4bdf24f17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:07.018779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb101e4-b418-4280-8295-56a4bdf24f17 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:07.039450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 689/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:09.188444 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f4bfa-6435-411d-bd17-d87f5f4ddf6a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:09.189002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f4bfa-6435-411d-bd17-d87f5f4ddf6a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:09.189287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f4bfa-6435-411d-bd17-d87f5f4ddf6a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:09.190776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 672/1361] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:09.196600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7502bd5-7ace-4d9e-9958-29463cfa42fe None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:09.196948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7502bd5-7ace-4d9e-9958-29463cfa42fe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:09.197132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7502bd5-7ace-4d9e-9958-29463cfa42fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:09.281016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e7502bd5-7ace-4d9e-9958-29463cfa42fe None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:09.282454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e7502bd5-7ace-4d9e-9958-29463cfa42fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:09.326976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:09.397903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 690/1362] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:09.706076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f4bfa-6435-411d-bd17-d87f5f4ddf6a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:09.707600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-602f4bfa-6435-411d-bd17-d87f5f4ddf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:09.889167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f4bfa-6435-411d-bd17-d87f5f4ddf6a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:09.901162 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:10.027828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43ddc2-3db8-4700-8c08-8ad92017f017 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:10.027828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43ddc2-3db8-4700-8c08-8ad92017f017 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:10.027828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43ddc2-3db8-4700-8c08-8ad92017f017 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:10.027828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43ddc2-3db8-4700-8c08-8ad92017f017 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:10.028352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd43ddc2-3db8-4700-8c08-8ad92017f017 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:10.133224 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43ddc2-3db8-4700-8c08-8ad92017f017 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:10.135028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd43ddc2-3db8-4700-8c08-8ad92017f017 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:10.154717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 673/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:12.478020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74051751-eae9-4814-bd43-30e14fdcceca None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:12.478953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74051751-eae9-4814-bd43-30e14fdcceca None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:12.479420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74051751-eae9-4814-bd43-30e14fdcceca None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:12.480822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 691/1364] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:12.486670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3305a3-682e-4961-b1a8-6c2ba2f88df8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:12.487180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3305a3-682e-4961-b1a8-6c2ba2f88df8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:12.487434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3305a3-682e-4961-b1a8-6c2ba2f88df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:12.510504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:12.580777 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6c3305a3-682e-4961-b1a8-6c2ba2f88df8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:12.582205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6c3305a3-682e-4961-b1a8-6c2ba2f88df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:12.698479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 674/1365] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:13.007784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adedb2f-5dd4-4b39-b95b-525ab44458fe None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:13.008375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adedb2f-5dd4-4b39-b95b-525ab44458fe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:13.008696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adedb2f-5dd4-4b39-b95b-525ab44458fe None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:13.010349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 692/1366] 10.222.0.16 () {54 vars in 962 bytes} [Fri Aug 19 21:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:13.016506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda36126-3719-4b3d-85ea-5eb1f46e4fd3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:13.016787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda36126-3719-4b3d-85ea-5eb1f46e4fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:13.017097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda36126-3719-4b3d-85ea-5eb1f46e4fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:13.120840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cda36126-3719-4b3d-85ea-5eb1f46e4fd3 None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:13.122374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cda36126-3719-4b3d-85ea-5eb1f46e4fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:13.169712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:13.289756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 675/1367] 10.222.0.16 () {58 vars in 1059 bytes} [Fri Aug 19 21:01:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:01:13.298205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adedb2f-5dd4-4b39-b95b-525ab44458fe None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:13.300045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0adedb2f-5dd4-4b39-b95b-525ab44458fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:13.478400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adedb2f-5dd4-4b39-b95b-525ab44458fe None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:13.501375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:13.605616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c8121-326e-439b-ae43-93f8a47d3f58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:13.606507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c8121-326e-439b-ae43-93f8a47d3f58 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:13.606885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c8121-326e-439b-ae43-93f8a47d3f58 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:13.607271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c8121-326e-439b-ae43-93f8a47d3f58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:13.609108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-132c8121-326e-439b-ae43-93f8a47d3f58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:13.709787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c8121-326e-439b-ae43-93f8a47d3f58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:13.711820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c8121-326e-439b-ae43-93f8a47d3f58 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:13.734659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 693/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:14.313840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54355c08-ad66-4761-a083-e5b6b052b8a2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:14.314394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54355c08-ad66-4761-a083-e5b6b052b8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:14.314394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54355c08-ad66-4761-a083-e5b6b052b8a2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:14.315995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 676/1369] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:14.322006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33cbce-2b66-41d7-bc00-1200a89d21b3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:14.322338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33cbce-2b66-41d7-bc00-1200a89d21b3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:14.322568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db33cbce-2b66-41d7-bc00-1200a89d21b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:14.359371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:14.401045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-db33cbce-2b66-41d7-bc00-1200a89d21b3 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:14.402425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-db33cbce-2b66-41d7-bc00-1200a89d21b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:14.513363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 694/1370] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:15.126687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1fdab-a95c-423a-866a-bb5db91c8f40 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:15.126687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1fdab-a95c-423a-866a-bb5db91c8f40 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:15.126687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1fdab-a95c-423a-866a-bb5db91c8f40 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:15.129369 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 677/1371] 10.222.0.16 () {54 vars in 964 bytes} [Fri Aug 19 21:01:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:15.132973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf241a-df1f-4721-aacf-e4da307993e0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:15.133331 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf241a-df1f-4721-aacf-e4da307993e0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:15.133674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf241a-df1f-4721-aacf-e4da307993e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:15.226170 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f0cf241a-df1f-4721-aacf-e4da307993e0 None None] MFA Rules not processed for user `54d1ae400b5c4ae0955047b9e7750103`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:15.227510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f0cf241a-df1f-4721-aacf-e4da307993e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:15.293512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:15.328593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 695/1372] 10.222.0.16 () {58 vars in 1061 bytes} [Fri Aug 19 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:15.336991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1fdab-a95c-423a-866a-bb5db91c8f40 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:15.338618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f2a1fdab-a95c-423a-866a-bb5db91c8f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:15.455561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1fdab-a95c-423a-866a-bb5db91c8f40 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:15.501595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:15.542810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5c9f99-fb58-400c-9f93-2c2dab99eb14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:15.543429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c9f99-fb58-400c-9f93-2c2dab99eb14 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:15.543808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c9f99-fb58-400c-9f93-2c2dab99eb14 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:15.543973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5c9f99-fb58-400c-9f93-2c2dab99eb14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:15.545655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8f5c9f99-fb58-400c-9f93-2c2dab99eb14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:15.714893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c9f99-fb58-400c-9f93-2c2dab99eb14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:15.717972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5c9f99-fb58-400c-9f93-2c2dab99eb14 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:15.751770 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 678/1373] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:15.792125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcd36a-d48c-4dc2-b5c7-0fd825f4c071 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:15.792125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcd36a-d48c-4dc2-b5c7-0fd825f4c071 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:15.792125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fcd36a-d48c-4dc2-b5c7-0fd825f4c071 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:15.797157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 696/1374] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:15.799538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98f07e-eb2f-4551-a6cc-3ac7f3d9090f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:15.800089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98f07e-eb2f-4551-a6cc-3ac7f3d9090f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:15.800684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd98f07e-eb2f-4551-a6cc-3ac7f3d9090f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:15.916149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bd98f07e-eb2f-4551-a6cc-3ac7f3d9090f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:15.917589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd98f07e-eb2f-4551-a6cc-3ac7f3d9090f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:15.940416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:16.029742 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 679/1375] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:01:16.332085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcd36a-d48c-4dc2-b5c7-0fd825f4c071 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:16.334372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-43fcd36a-d48c-4dc2-b5c7-0fd825f4c071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:16.488175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fcd36a-d48c-4dc2-b5c7-0fd825f4c071 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:16.541963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:16.606615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb52e25-cb25-4e4b-9480-11f0456e28c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:16.607142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb52e25-cb25-4e4b-9480-11f0456e28c7 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:16.607423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb52e25-cb25-4e4b-9480-11f0456e28c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:16.607698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb52e25-cb25-4e4b-9480-11f0456e28c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:16.609316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8cb52e25-cb25-4e4b-9480-11f0456e28c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:16.702609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb52e25-cb25-4e4b-9480-11f0456e28c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:16.704243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb52e25-cb25-4e4b-9480-11f0456e28c7 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:16.724680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 697/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:18.746608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec45b3d-0d61-4890-8fc5-e3e129e92198 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:18.747441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec45b3d-0d61-4890-8fc5-e3e129e92198 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:18.747441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec45b3d-0d61-4890-8fc5-e3e129e92198 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:18.748744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 680/1377] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:18.754534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0ba1b-1b9b-458f-be57-68fccff914c7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:18.755106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0ba1b-1b9b-458f-be57-68fccff914c7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:18.755206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0ba1b-1b9b-458f-be57-68fccff914c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:18.840747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-add0ba1b-1b9b-458f-be57-68fccff914c7 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:18.842269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-add0ba1b-1b9b-458f-be57-68fccff914c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:18.851389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:18.968371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 698/1378] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:19.271865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec45b3d-0d61-4890-8fc5-e3e129e92198 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:19.273794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ec45b3d-0d61-4890-8fc5-e3e129e92198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:19.478696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec45b3d-0d61-4890-8fc5-e3e129e92198 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:19.487648 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:19.614466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d77893-c2a2-4f74-a3d4-80612ef119c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:19.614849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d77893-c2a2-4f74-a3d4-80612ef119c2 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:19.615362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d77893-c2a2-4f74-a3d4-80612ef119c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:19.615804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d77893-c2a2-4f74-a3d4-80612ef119c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:19.617727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-63d77893-c2a2-4f74-a3d4-80612ef119c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:19.727319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d77893-c2a2-4f74-a3d4-80612ef119c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:19.729109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d77893-c2a2-4f74-a3d4-80612ef119c2 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:19.749972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 681/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:21.761450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e863b5c-e55f-44f7-b7d4-4872d7858665 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:21.762044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e863b5c-e55f-44f7-b7d4-4872d7858665 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:21.762044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e863b5c-e55f-44f7-b7d4-4872d7858665 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:21.763564 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 699/1380] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:21.769025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e03ed5-3c7e-4ed4-94c4-e958dbcfb438 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:21.769499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e03ed5-3c7e-4ed4-94c4-e958dbcfb438 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:21.769608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e03ed5-3c7e-4ed4-94c4-e958dbcfb438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:21.781896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:21.844939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-39e03ed5-3c7e-4ed4-94c4-e958dbcfb438 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:21.846505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-39e03ed5-3c7e-4ed4-94c4-e958dbcfb438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:21.957474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 682/1381] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:22.266269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e863b5c-e55f-44f7-b7d4-4872d7858665 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:22.268278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3e863b5c-e55f-44f7-b7d4-4872d7858665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:22.435859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:22.447932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e863b5c-e55f-44f7-b7d4-4872d7858665 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:22.579237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9693e5b-dcb8-4729-84f1-d4365d64cd2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:22.580002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9693e5b-dcb8-4729-84f1-d4365d64cd2f service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:22.580426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9693e5b-dcb8-4729-84f1-d4365d64cd2f service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:22.580709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9693e5b-dcb8-4729-84f1-d4365d64cd2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:22.582355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d9693e5b-dcb8-4729-84f1-d4365d64cd2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:22.692638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9693e5b-dcb8-4729-84f1-d4365d64cd2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:22.695016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9693e5b-dcb8-4729-84f1-d4365d64cd2f service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:22.707179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:22.718551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 700/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:24.988357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ffbdd-f33c-4ebb-9dbb-f37bae279b1d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:24.989077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ffbdd-f33c-4ebb-9dbb-f37bae279b1d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:24.989077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ffbdd-f33c-4ebb-9dbb-f37bae279b1d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:24.990933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 683/1383] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 21:01:24.996565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e481385-f5a5-417a-87b6-e6254d73a6fa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:24.997477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e481385-f5a5-417a-87b6-e6254d73a6fa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:24.997477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e481385-f5a5-417a-87b6-e6254d73a6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:25.089954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3e481385-f5a5-417a-87b6-e6254d73a6fa None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:25.091252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3e481385-f5a5-417a-87b6-e6254d73a6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:25.200335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 701/1384] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:25.581884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ffbdd-f33c-4ebb-9dbb-f37bae279b1d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:25.583489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-387ffbdd-f33c-4ebb-9dbb-f37bae279b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:25.707359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:25.789187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ffbdd-f33c-4ebb-9dbb-f37bae279b1d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:25.938643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88085d80-707a-4c95-90b1-5dc24e4c8c0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:25.939265 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88085d80-707a-4c95-90b1-5dc24e4c8c0f service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:25.939722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88085d80-707a-4c95-90b1-5dc24e4c8c0f service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:25.939842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88085d80-707a-4c95-90b1-5dc24e4c8c0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:25.941461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88085d80-707a-4c95-90b1-5dc24e4c8c0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:25.995769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:26.080429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88085d80-707a-4c95-90b1-5dc24e4c8c0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:26.084995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88085d80-707a-4c95-90b1-5dc24e4c8c0f service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:26.109842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 684/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:28.696375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47ac9f-1249-4e6a-ab2e-1e5b5592ed26 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:28.696914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47ac9f-1249-4e6a-ab2e-1e5b5592ed26 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:28.697075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47ac9f-1249-4e6a-ab2e-1e5b5592ed26 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:28.698484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 702/1386] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:28.704576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89848996-dc46-410e-b22a-a95095a1f393 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:28.705043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89848996-dc46-410e-b22a-a95095a1f393 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:28.705340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89848996-dc46-410e-b22a-a95095a1f393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:28.797227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-89848996-dc46-410e-b22a-a95095a1f393 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:28.798623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-89848996-dc46-410e-b22a-a95095a1f393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:28.916260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 685/1387] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:29.210383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47ac9f-1249-4e6a-ab2e-1e5b5592ed26 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:29.211981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1d47ac9f-1249-4e6a-ab2e-1e5b5592ed26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:29.299142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:29.381091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47ac9f-1249-4e6a-ab2e-1e5b5592ed26 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:29.501070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba51326-5e64-4bfa-b23b-62f525c63195 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:29.501678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51326-5e64-4bfa-b23b-62f525c63195 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:29.501985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51326-5e64-4bfa-b23b-62f525c63195 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:29.502256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba51326-5e64-4bfa-b23b-62f525c63195 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:29.503856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8ba51326-5e64-4bfa-b23b-62f525c63195 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:29.544079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:29.599355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba51326-5e64-4bfa-b23b-62f525c63195 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:29.600982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba51326-5e64-4bfa-b23b-62f525c63195 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:29.621088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 703/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:31.091676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27c86a-ac7a-4105-9af2-4900c7a9d0aa None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:31.092093 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27c86a-ac7a-4105-9af2-4900c7a9d0aa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:31.092238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27c86a-ac7a-4105-9af2-4900c7a9d0aa None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:31.093641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 686/1389] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:31.099128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc117a8-5344-4816-bbd5-6a9d2c1e9b92 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:31.099593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc117a8-5344-4816-bbd5-6a9d2c1e9b92 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:31.099700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc117a8-5344-4816-bbd5-6a9d2c1e9b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:31.171446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ecc117a8-5344-4816-bbd5-6a9d2c1e9b92 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:31.172696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ecc117a8-5344-4816-bbd5-6a9d2c1e9b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:31.280690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 704/1390] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:31.588915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b101-e504-40c5-ad23-f89b0e3e7e53 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:31.589299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b101-e504-40c5-ad23-f89b0e3e7e53 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:31.589649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b101-e504-40c5-ad23-f89b0e3e7e53 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:31.591067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 687/1391] 10.222.0.16 () {54 vars in 967 bytes} [Fri Aug 19 21:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:31.591197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf13573-bb1b-442c-9d15-a45e970bb25f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:31.591423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf13573-bb1b-442c-9d15-a45e970bb25f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:31.591701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf13573-bb1b-442c-9d15-a45e970bb25f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:31.593133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 705/1392] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 21:01:31.596110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740677d-6327-4903-80da-2cc041a22263 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:31.596457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740677d-6327-4903-80da-2cc041a22263 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:31.596801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c740677d-6327-4903-80da-2cc041a22263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:31.598658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e044b3-80f3-4a8f-8449-5873d85eed0f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:31.599538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e044b3-80f3-4a8f-8449-5873d85eed0f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:31.599718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e044b3-80f3-4a8f-8449-5873d85eed0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:31.615674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:31.631472 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:31.677687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-05e044b3-80f3-4a8f-8449-5873d85eed0f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:31.679034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-05e044b3-80f3-4a8f-8449-5873d85eed0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:31.685130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c740677d-6327-4903-80da-2cc041a22263 None None] MFA Rules not processed for user `b957b367ddf2483f84056ccec57fc515`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:31.686409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c740677d-6327-4903-80da-2cc041a22263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:31.766685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 688/1393] 10.222.0.16 () {58 vars in 1064 bytes} [Fri Aug 19 21:01:31 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:01:31.775138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740677d-6327-4903-80da-2cc041a22263 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:31.782638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c740677d-6327-4903-80da-2cc041a22263 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:31.784815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 706/1394] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:01:31.862479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc1f40-70ef-451b-ae60-fcf07b50b145 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b957b367ddf2483f84056ccec57fc515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:31.863108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1f40-70ef-451b-ae60-fcf07b50b145 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:31.863393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1f40-70ef-451b-ae60-fcf07b50b145 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:31.863682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc1f40-70ef-451b-ae60-fcf07b50b145 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:31.865368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4acc1f40-70ef-451b-ae60-fcf07b50b145 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:31.910552 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:31.982312 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc1f40-70ef-451b-ae60-fcf07b50b145 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:31.983676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc1f40-70ef-451b-ae60-fcf07b50b145 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:32.003253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 689/1395] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:01:32.082845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e044b3-80f3-4a8f-8449-5873d85eed0f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:32.084672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-05e044b3-80f3-4a8f-8449-5873d85eed0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:32.176798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:32.204974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e044b3-80f3-4a8f-8449-5873d85eed0f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:32.340187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3be10-cf7a-4587-bb16-0e2365be3337 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:32.340809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3be10-cf7a-4587-bb16-0e2365be3337 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:32.341087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3be10-cf7a-4587-bb16-0e2365be3337 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:32.341358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3be10-cf7a-4587-bb16-0e2365be3337 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:32.343093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04a3be10-cf7a-4587-bb16-0e2365be3337 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:32.468917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:32.471717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3be10-cf7a-4587-bb16-0e2365be3337 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:32.474807 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3be10-cf7a-4587-bb16-0e2365be3337 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:32.498156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 707/1396] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:32.795446 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaea358-ba35-41c7-b31b-ab00df495322 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:32.795960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaea358-ba35-41c7-b31b-ab00df495322 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:32.796057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaea358-ba35-41c7-b31b-ab00df495322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:32.871599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-daaea358-ba35-41c7-b31b-ab00df495322 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:32.873069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-daaea358-ba35-41c7-b31b-ab00df495322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:32.966851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 690/1397] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:01:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:33.012511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f089ef-5c76-414f-8853-34d418ab78fe None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:33.012847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f089ef-5c76-414f-8853-34d418ab78fe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:33.013077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f089ef-5c76-414f-8853-34d418ab78fe None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:33.014480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 708/1398] 10.222.0.16 () {54 vars in 943 bytes} [Fri Aug 19 21:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:33.021747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4939a-21d5-40a0-94ec-54cfbcdc336a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:33.022259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4939a-21d5-40a0-94ec-54cfbcdc336a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:33.022708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4939a-21d5-40a0-94ec-54cfbcdc336a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:33.035883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:33.108628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dbe4939a-21d5-40a0-94ec-54cfbcdc336a None None] MFA Rules not processed for user `61a0906b8b134c178b4dece2891f1596`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:33.109891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dbe4939a-21d5-40a0-94ec-54cfbcdc336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:33.227995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 691/1399] 10.222.0.16 () {58 vars in 1040 bytes} [Fri Aug 19 21:01:33 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:33.237601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f089ef-5c76-414f-8853-34d418ab78fe None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:33.239311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e5f089ef-5c76-414f-8853-34d418ab78fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:33.397741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f089ef-5c76-414f-8853-34d418ab78fe None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:33.477380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:33.516931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7926f87a-cb4b-462a-8b4c-1881c11629f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:33.517559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f87a-cb4b-462a-8b4c-1881c11629f4 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:33.517865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f87a-cb4b-462a-8b4c-1881c11629f4 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:33.518149 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7926f87a-cb4b-462a-8b4c-1881c11629f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:33.519898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7926f87a-cb4b-462a-8b4c-1881c11629f4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:33.613311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926f87a-cb4b-462a-8b4c-1881c11629f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:33.614962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7926f87a-cb4b-462a-8b4c-1881c11629f4 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:33.623616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 709/1400] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:01:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:01:34.515195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef233fe3-7c12-4ef6-be1e-4d7c3813f83d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:34.515734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef233fe3-7c12-4ef6-be1e-4d7c3813f83d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:34.515734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef233fe3-7c12-4ef6-be1e-4d7c3813f83d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:34.517405 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 692/1401] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:34.524202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef968e4c-13cc-4b80-95c1-5a267c592814 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:34.524715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef968e4c-13cc-4b80-95c1-5a267c592814 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:34.524796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef968e4c-13cc-4b80-95c1-5a267c592814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:34.610276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ef968e4c-13cc-4b80-95c1-5a267c592814 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:34.611518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ef968e4c-13cc-4b80-95c1-5a267c592814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:34.681130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:34.727213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 710/1402] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:35.020860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef233fe3-7c12-4ef6-be1e-4d7c3813f83d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:35.022588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ef233fe3-7c12-4ef6-be1e-4d7c3813f83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:35.116208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:35.204854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef233fe3-7c12-4ef6-be1e-4d7c3813f83d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:35.342121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aa5e9-5780-4e31-b96d-845f12095ec3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:35.342699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa5e9-5780-4e31-b96d-845f12095ec3 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:35.342973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa5e9-5780-4e31-b96d-845f12095ec3 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:35.343255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa5e9-5780-4e31-b96d-845f12095ec3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:35.345040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-493aa5e9-5780-4e31-b96d-845f12095ec3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:35.384076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:35.462766 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493aa5e9-5780-4e31-b96d-845f12095ec3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:35.463752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493aa5e9-5780-4e31-b96d-845f12095ec3 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:35.484395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 693/1403] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:01:37.536748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016197be-46c1-4a0a-a2f9-dbd14c534032 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:37.537513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016197be-46c1-4a0a-a2f9-dbd14c534032 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:37.537513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016197be-46c1-4a0a-a2f9-dbd14c534032 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:37.542186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 711/1404] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:37.544320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dccc8-87b7-4062-9690-aa71a2269c0d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:37.544951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dccc8-87b7-4062-9690-aa71a2269c0d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:37.544951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1dccc8-87b7-4062-9690-aa71a2269c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:37.631725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3f1dccc8-87b7-4062-9690-aa71a2269c0d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:37.632964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3f1dccc8-87b7-4062-9690-aa71a2269c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:37.771421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 694/1405] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:38.066526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016197be-46c1-4a0a-a2f9-dbd14c534032 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:38.068079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-016197be-46c1-4a0a-a2f9-dbd14c534032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:38.245850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016197be-46c1-4a0a-a2f9-dbd14c534032 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:38.299788 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:38.378297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67a1f7-930c-438f-84d2-45132d19082e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:38.378826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67a1f7-930c-438f-84d2-45132d19082e service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:38.379128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67a1f7-930c-438f-84d2-45132d19082e service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:38.379421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67a1f7-930c-438f-84d2-45132d19082e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:38.381922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ef67a1f7-930c-438f-84d2-45132d19082e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:38.490943 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67a1f7-930c-438f-84d2-45132d19082e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:38.492564 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67a1f7-930c-438f-84d2-45132d19082e service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:38.512687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 712/1406] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:40.525313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb1a90-ae67-491c-b374-92fa4e3bf05b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:40.526190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb1a90-ae67-491c-b374-92fa4e3bf05b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:40.526190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb1a90-ae67-491c-b374-92fa4e3bf05b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:40.527233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 695/1407] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:40.532896 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae36dea-a62b-43bd-acb2-c2ea55f6bb98 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:40.533360 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae36dea-a62b-43bd-acb2-c2ea55f6bb98 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:40.533453 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae36dea-a62b-43bd-acb2-c2ea55f6bb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:40.615794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:40.621743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5ae36dea-a62b-43bd-acb2-c2ea55f6bb98 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:40.624296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5ae36dea-a62b-43bd-acb2-c2ea55f6bb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:40.772824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 713/1408] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:41.072426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb1a90-ae67-491c-b374-92fa4e3bf05b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:41.072426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88fb1a90-ae67-491c-b374-92fa4e3bf05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:41.080572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:41.301825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb1a90-ae67-491c-b374-92fa4e3bf05b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:41.406334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:41.448629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d459b1c-239c-4a9f-9b35-02c1cf611f93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:41.449614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d459b1c-239c-4a9f-9b35-02c1cf611f93 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:41.449931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d459b1c-239c-4a9f-9b35-02c1cf611f93 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:41.450200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d459b1c-239c-4a9f-9b35-02c1cf611f93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:41.452033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3d459b1c-239c-4a9f-9b35-02c1cf611f93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:41.561536 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d459b1c-239c-4a9f-9b35-02c1cf611f93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:01:41.563242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d459b1c-239c-4a9f-9b35-02c1cf611f93 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:01:41.583626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 696/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:01:50.765720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=43 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:51.580238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=59 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:56.200906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0c9be-2f66-4983-8438-aa92ee8fa869 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:56.201528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0c9be-2f66-4983-8438-aa92ee8fa869 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:56.201528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0c9be-2f66-4983-8438-aa92ee8fa869 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:56.203425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 714/1410] 10.222.0.16 () {54 vars in 918 bytes} [Fri Aug 19 21:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:56.211134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbaf64-c5d1-47a3-86d7-89511d84ddc2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:56.211626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbaf64-c5d1-47a3-86d7-89511d84ddc2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:56.211743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bbaf64-c5d1-47a3-86d7-89511d84ddc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:56.289451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-41bbaf64-c5d1-47a3-86d7-89511d84ddc2 None None] MFA Rules not processed for user `824165e55d2f4742aa435f1edff9b403`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:56.290751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-41bbaf64-c5d1-47a3-86d7-89511d84ddc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:56.400360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 697/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Aug 19 21:01:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:56.415243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ccbe9-6925-4c56-a848-1b9691dd0f2c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:56.415530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ccbe9-6925-4c56-a848-1b9691dd0f2c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:56.415808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ccbe9-6925-4c56-a848-1b9691dd0f2c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:56.417163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 715/1412] 10.222.0.16 () {54 vars in 918 bytes} [Fri Aug 19 21:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:56.422004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166de1-ed4d-4349-b603-0f690ca1bfa7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:56.422292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166de1-ed4d-4349-b603-0f690ca1bfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:56.422565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73166de1-ed4d-4349-b603-0f690ca1bfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:56.499812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:56.502703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-73166de1-ed4d-4349-b603-0f690ca1bfa7 None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:56.504479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-73166de1-ed4d-4349-b603-0f690ca1bfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:56.629372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 698/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Aug 19 21:01:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:59.087056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c038b07-3566-4a08-a106-7d3bf8c67176 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:59.087634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c038b07-3566-4a08-a106-7d3bf8c67176 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:59.087634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c038b07-3566-4a08-a106-7d3bf8c67176 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:59.088935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:01:59.094365 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebd5d1-3e50-444d-b432-ad3289d19439 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:59.094765 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebd5d1-3e50-444d-b432-ad3289d19439 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:59.094926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebd5d1-3e50-444d-b432-ad3289d19439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:59.165850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-02ebd5d1-3e50-444d-b432-ad3289d19439 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:01:59.167085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-02ebd5d1-3e50-444d-b432-ad3289d19439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:59.226594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:59.271694 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:01:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:01:59.595558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c038b07-3566-4a08-a106-7d3bf8c67176 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:01:59.597180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9c038b07-3566-4a08-a106-7d3bf8c67176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:01:59.773845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c038b07-3566-4a08-a106-7d3bf8c67176 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:01:59.878825 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:01:59.931306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec9bb6-ab26-4eb9-b8f4-a5700354f81e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:01:59.931934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec9bb6-ab26-4eb9-b8f4-a5700354f81e service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:01:59.932212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec9bb6-ab26-4eb9-b8f4-a5700354f81e service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:01:59.932481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec9bb6-ab26-4eb9-b8f4-a5700354f81e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:01:59.934176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aeec9bb6-ab26-4eb9-b8f4-a5700354f81e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:00.046141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec9bb6-ab26-4eb9-b8f4-a5700354f81e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:00.047789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec9bb6-ab26-4eb9-b8f4-a5700354f81e service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:00.070699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 717/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:01:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:01.101757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07e4c5-f775-47e4-a3fd-0e562793a91e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:01.102221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07e4c5-f775-47e4-a3fd-0e562793a91e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:01.102338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07e4c5-f775-47e4-a3fd-0e562793a91e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:01.103706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:01.108644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e8e9-044f-4fe4-bdad-765115e2f6fe None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:01.109068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e8e9-044f-4fe4-bdad-765115e2f6fe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:01.109169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c857e8e9-044f-4fe4-bdad-765115e2f6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:01.193395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c857e8e9-044f-4fe4-bdad-765115e2f6fe None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:01.194681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c857e8e9-044f-4fe4-bdad-765115e2f6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:01.227244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:01.300220 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:01.459762 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07e4c5-f775-47e4-a3fd-0e562793a91e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:01.461328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd07e4c5-f775-47e4-a3fd-0e562793a91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:01.612262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07e4c5-f775-47e4-a3fd-0e562793a91e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:01.653677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:01.735737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5013b3-21d1-4abf-b67d-7fe941ec53d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:01.736352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5013b3-21d1-4abf-b67d-7fe941ec53d2 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:01.736631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5013b3-21d1-4abf-b67d-7fe941ec53d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:01.736903 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5013b3-21d1-4abf-b67d-7fe941ec53d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:01.738556 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa5013b3-21d1-4abf-b67d-7fe941ec53d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:01.830391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5013b3-21d1-4abf-b67d-7fe941ec53d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:01.832026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5013b3-21d1-4abf-b67d-7fe941ec53d2 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:01.852019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 701/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:02.752575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143207f-2578-46ac-ad04-c82178d7a185 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:02.753167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143207f-2578-46ac-ad04-c82178d7a185 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:02.753167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143207f-2578-46ac-ad04-c82178d7a185 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:02.755304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:02.761335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd9452-e6dc-4fd6-bd00-ffc6466a8028 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:02.761879 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd9452-e6dc-4fd6-bd00-ffc6466a8028 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:02.761981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd9452-e6dc-4fd6-bd00-ffc6466a8028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:02.818935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:02.844561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-65fd9452-e6dc-4fd6-bd00-ffc6466a8028 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:02.846114 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-65fd9452-e6dc-4fd6-bd00-ffc6466a8028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:03.065692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:03.223078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143207f-2578-46ac-ad04-c82178d7a185 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:03.225734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f143207f-2578-46ac-ad04-c82178d7a185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:03.387266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143207f-2578-46ac-ad04-c82178d7a185 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:03.399984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:03.506310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f2efa-af86-45f2-a877-e7a36913b26e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:03.506936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f2efa-af86-45f2-a877-e7a36913b26e service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:03.507211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f2efa-af86-45f2-a877-e7a36913b26e service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:03.507482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f2efa-af86-45f2-a877-e7a36913b26e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:03.509125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fa2f2efa-af86-45f2-a877-e7a36913b26e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:03.604919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f2efa-af86-45f2-a877-e7a36913b26e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:03.606634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f2efa-af86-45f2-a877-e7a36913b26e service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:03.629056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 720/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:04.509495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e03e34-410c-45bf-9685-4f0b2a3d12bb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:04.510112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e03e34-410c-45bf-9685-4f0b2a3d12bb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:04.510112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e03e34-410c-45bf-9685-4f0b2a3d12bb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:04.511398 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:04.516457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea71a5-7e40-4cde-8786-28ec4a3c1b17 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:04.516971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea71a5-7e40-4cde-8786-28ec4a3c1b17 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:04.517076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aea71a5-7e40-4cde-8786-28ec4a3c1b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:04.537994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:04.598033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1aea71a5-7e40-4cde-8786-28ec4a3c1b17 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:04.599865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1aea71a5-7e40-4cde-8786-28ec4a3c1b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:04.699952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:04.854571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e03e34-410c-45bf-9685-4f0b2a3d12bb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:04.856297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-12e03e34-410c-45bf-9685-4f0b2a3d12bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:04.977248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:05.016297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e03e34-410c-45bf-9685-4f0b2a3d12bb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:05.141157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b3d6be-0a6f-4100-9c66-00e5eb147da2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:05.141786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3d6be-0a6f-4100-9c66-00e5eb147da2 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:05.142078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3d6be-0a6f-4100-9c66-00e5eb147da2 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:05.142354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b3d6be-0a6f-4100-9c66-00e5eb147da2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:05.143967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7b3d6be-0a6f-4100-9c66-00e5eb147da2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:05.221628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:05.236600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3d6be-0a6f-4100-9c66-00e5eb147da2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:05.238267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b3d6be-0a6f-4100-9c66-00e5eb147da2 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:05.257891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 704/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:06.136670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ad272-4e09-4c51-a87d-d978ab14c03a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:06.137259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ad272-4e09-4c51-a87d-d978ab14c03a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:06.137259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ad272-4e09-4c51-a87d-d978ab14c03a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:06.138663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:06.143773 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5565c-f5af-4d15-8018-1d8fac169eb2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:06.144215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5565c-f5af-4d15-8018-1d8fac169eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:06.144330 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5565c-f5af-4d15-8018-1d8fac169eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:06.224585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-07a5565c-f5af-4d15-8018-1d8fac169eb2 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:06.225857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-07a5565c-f5af-4d15-8018-1d8fac169eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:06.331737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:06.488177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ad272-4e09-4c51-a87d-d978ab14c03a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:06.490359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6b0ad272-4e09-4c51-a87d-d978ab14c03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:06.583144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:06.649732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ad272-4e09-4c51-a87d-d978ab14c03a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:06.765831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5369d0-cbc7-4577-ba69-4bf4adc500a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:06.766441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5369d0-cbc7-4577-ba69-4bf4adc500a0 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:06.766723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5369d0-cbc7-4577-ba69-4bf4adc500a0 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:06.767295 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5369d0-cbc7-4577-ba69-4bf4adc500a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:06.769351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fd5369d0-cbc7-4577-ba69-4bf4adc500a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:06.821539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:06.866438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5369d0-cbc7-4577-ba69-4bf4adc500a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:06.868058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5369d0-cbc7-4577-ba69-4bf4adc500a0 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:06.887460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 723/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:07.717181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc990b-78ee-4090-a393-5b0e81b604dc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:07.717739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc990b-78ee-4090-a393-5b0e81b604dc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:07.717739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc990b-78ee-4090-a393-5b0e81b604dc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:07.719072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:02:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:07.724217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c9e69-100c-4f7c-a14d-f6243cc9297a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:07.724650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c9e69-100c-4f7c-a14d-f6243cc9297a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:07.724757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c9e69-100c-4f7c-a14d-f6243cc9297a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:07.804004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-277c9e69-100c-4f7c-a14d-f6243cc9297a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:07.805241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-277c9e69-100c-4f7c-a14d-f6243cc9297a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:07.908306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:08.076262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc990b-78ee-4090-a393-5b0e81b604dc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:08.077836 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-70bc990b-78ee-4090-a393-5b0e81b604dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:08.128970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:08.227750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc990b-78ee-4090-a393-5b0e81b604dc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:08.345912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a104a54-8c83-4db6-b36d-aa52533cbc40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:08.346817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a104a54-8c83-4db6-b36d-aa52533cbc40 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:08.347306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a104a54-8c83-4db6-b36d-aa52533cbc40 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:08.347612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a104a54-8c83-4db6-b36d-aa52533cbc40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:08.349319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4a104a54-8c83-4db6-b36d-aa52533cbc40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:08.356670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:08.440324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a104a54-8c83-4db6-b36d-aa52533cbc40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:08.442471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a104a54-8c83-4db6-b36d-aa52533cbc40 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:08.462587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 707/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:09.287128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f05ab7-0000-4000-bec0-37b0475a43d4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:09.287724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f05ab7-0000-4000-bec0-37b0475a43d4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:09.287724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f05ab7-0000-4000-bec0-37b0475a43d4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:09.289040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:09.294509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ae7b0-7a7c-46ac-acfc-50ed0a96124d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:09.295041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ae7b0-7a7c-46ac-acfc-50ed0a96124d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:09.295149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38ae7b0-7a7c-46ac-acfc-50ed0a96124d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:09.380755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f38ae7b0-7a7c-46ac-acfc-50ed0a96124d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:09.381990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f38ae7b0-7a7c-46ac-acfc-50ed0a96124d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:09.451014 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:09.483392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:09.641445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f05ab7-0000-4000-bec0-37b0475a43d4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:09.642933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a0f05ab7-0000-4000-bec0-37b0475a43d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:09.661706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:09.795769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f05ab7-0000-4000-bec0-37b0475a43d4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:09.887011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:09.915393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec2f54-7c3c-450e-95ef-4db36174ed78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:09.915921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2f54-7c3c-450e-95ef-4db36174ed78 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:09.916199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2f54-7c3c-450e-95ef-4db36174ed78 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:09.916472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec2f54-7c3c-450e-95ef-4db36174ed78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:09.918120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0fec2f54-7c3c-450e-95ef-4db36174ed78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:10.009117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec2f54-7c3c-450e-95ef-4db36174ed78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:10.010771 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec2f54-7c3c-450e-95ef-4db36174ed78 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:10.030364 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 726/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:10.844922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898deb28-a8c1-4811-b7a8-565f2f42773c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:10.845810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898deb28-a8c1-4811-b7a8-565f2f42773c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:10.845810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898deb28-a8c1-4811-b7a8-565f2f42773c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:10.846941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:10.852304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b06b26-4311-4cae-9334-25d88b681ca8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:10.852759 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b06b26-4311-4cae-9334-25d88b681ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:10.852870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b06b26-4311-4cae-9334-25d88b681ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:10.938738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e0b06b26-4311-4cae-9334-25d88b681ca8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:10.939962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e0b06b26-4311-4cae-9334-25d88b681ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:10.983341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:11.040915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:11.196685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898deb28-a8c1-4811-b7a8-565f2f42773c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:11.198322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-898deb28-a8c1-4811-b7a8-565f2f42773c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:11.352663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898deb28-a8c1-4811-b7a8-565f2f42773c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:11.407930 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:11.472284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9ad38b-75d2-4155-9fb9-daf6a4f2822c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:11.472879 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ad38b-75d2-4155-9fb9-daf6a4f2822c service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:11.473223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ad38b-75d2-4155-9fb9-daf6a4f2822c service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:11.474155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9ad38b-75d2-4155-9fb9-daf6a4f2822c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:11.475796 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2a9ad38b-75d2-4155-9fb9-daf6a4f2822c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:11.570786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ad38b-75d2-4155-9fb9-daf6a4f2822c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:11.571792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9ad38b-75d2-4155-9fb9-daf6a4f2822c service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:11.591040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 710/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:12.450295 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72616606-0fba-4738-95ee-622ef39a628f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:12.450871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72616606-0fba-4738-95ee-622ef39a628f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:12.450871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72616606-0fba-4738-95ee-622ef39a628f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:12.452190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:12.458819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935fc2-4609-49ea-8ea2-faf74d346deb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:12.459360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935fc2-4609-49ea-8ea2-faf74d346deb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:12.459453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7935fc2-4609-49ea-8ea2-faf74d346deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:12.555049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f7935fc2-4609-49ea-8ea2-faf74d346deb None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:12.556309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f7935fc2-4609-49ea-8ea2-faf74d346deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:12.558607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:12.659800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:12.811592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72616606-0fba-4738-95ee-622ef39a628f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:12.813154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-72616606-0fba-4738-95ee-622ef39a628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:12.967745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72616606-0fba-4738-95ee-622ef39a628f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:12.995677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:13.083842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04079b3c-fcd3-480a-9279-345bee9955a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:13.084367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04079b3c-fcd3-480a-9279-345bee9955a9 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:13.084638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04079b3c-fcd3-480a-9279-345bee9955a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:13.084907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04079b3c-fcd3-480a-9279-345bee9955a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:13.086608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04079b3c-fcd3-480a-9279-345bee9955a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:13.178408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04079b3c-fcd3-480a-9279-345bee9955a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:13.180021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04079b3c-fcd3-480a-9279-345bee9955a9 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:13.198212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 729/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:15.059943 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef24ce-7671-4d4f-9d7d-809929e073d4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:15.060516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef24ce-7671-4d4f-9d7d-809929e073d4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:15.060516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef24ce-7671-4d4f-9d7d-809929e073d4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:15.061932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:15.067789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fedcc2-2a3e-4419-b926-09075b1aaaf7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:15.068293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fedcc2-2a3e-4419-b926-09075b1aaaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:15.068407 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fedcc2-2a3e-4419-b926-09075b1aaaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:15.111680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:15.144261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-09fedcc2-2a3e-4419-b926-09075b1aaaf7 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:15.146398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-09fedcc2-2a3e-4419-b926-09075b1aaaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:15.249433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:02:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:15.546001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef24ce-7671-4d4f-9d7d-809929e073d4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:15.547581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-afef24ce-7671-4d4f-9d7d-809929e073d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:15.689196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef24ce-7671-4d4f-9d7d-809929e073d4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:15.727972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:15.776925 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20be860-4806-4ad2-bc35-c30978f34424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:15.777547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20be860-4806-4ad2-bc35-c30978f34424 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:15.777857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20be860-4806-4ad2-bc35-c30978f34424 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:15.778139 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20be860-4806-4ad2-bc35-c30978f34424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:15.779811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e20be860-4806-4ad2-bc35-c30978f34424 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:15.901862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20be860-4806-4ad2-bc35-c30978f34424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:15.903239 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20be860-4806-4ad2-bc35-c30978f34424 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:15.923002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 713/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:18.122239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52a3a-7383-4aae-9aa7-0dbb4be1dd65 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:18.123180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52a3a-7383-4aae-9aa7-0dbb4be1dd65 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:18.123180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e52a3a-7383-4aae-9aa7-0dbb4be1dd65 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:18.124482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:18.130368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87990dbd-575e-4f09-8718-02cb29acbe54 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:18.130852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87990dbd-575e-4f09-8718-02cb29acbe54 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:18.130974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87990dbd-575e-4f09-8718-02cb29acbe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:18.201639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-87990dbd-575e-4f09-8718-02cb29acbe54 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:18.202923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-87990dbd-575e-4f09-8718-02cb29acbe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:18.217315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:18.309305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:18.599423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e52a3a-7383-4aae-9aa7-0dbb4be1dd65 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:18.601022 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-64e52a3a-7383-4aae-9aa7-0dbb4be1dd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:18.710239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e52a3a-7383-4aae-9aa7-0dbb4be1dd65 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:18.723591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:18.790253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7957e5-f667-4a71-8345-6aba77083edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:18.790870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7957e5-f667-4a71-8345-6aba77083edf service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:18.791150 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7957e5-f667-4a71-8345-6aba77083edf service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:18.791423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7957e5-f667-4a71-8345-6aba77083edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:18.793054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a7957e5-f667-4a71-8345-6aba77083edf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:18.912660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7957e5-f667-4a71-8345-6aba77083edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:18.914045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7957e5-f667-4a71-8345-6aba77083edf service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:18.933147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 732/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:21.206161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87211f-9e98-406c-ae5e-d436082201b2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:21.206770 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87211f-9e98-406c-ae5e-d436082201b2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:21.206770 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a87211f-9e98-406c-ae5e-d436082201b2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:21.209034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:02:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:21.214897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e3a9b-004a-4d2c-8af4-cfe823c24fda None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:21.215362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e3a9b-004a-4d2c-8af4-cfe823c24fda None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:21.215476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e3a9b-004a-4d2c-8af4-cfe823c24fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:21.265638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:21.286394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5b0e3a9b-004a-4d2c-8af4-cfe823c24fda None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:21.287661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5b0e3a9b-004a-4d2c-8af4-cfe823c24fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:21.392142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:21.689152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87211f-9e98-406c-ae5e-d436082201b2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:21.690842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2a87211f-9e98-406c-ae5e-d436082201b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:21.802643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a87211f-9e98-406c-ae5e-d436082201b2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:21.843872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:21.881049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305d74a-4438-413d-bce5-6b27d5984dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:21.881675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305d74a-4438-413d-bce5-6b27d5984dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:21.881988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305d74a-4438-413d-bce5-6b27d5984dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:21.882262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305d74a-4438-413d-bce5-6b27d5984dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:21.883926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e305d74a-4438-413d-bce5-6b27d5984dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:21.995752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305d74a-4438-413d-bce5-6b27d5984dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:21.997122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305d74a-4438-413d-bce5-6b27d5984dd9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:22.024266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 716/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:24.090564 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1b601-28e4-4a8a-8f1e-8b6daf864a3b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:24.091058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1b601-28e4-4a8a-8f1e-8b6daf864a3b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:24.091181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1b601-28e4-4a8a-8f1e-8b6daf864a3b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:24.092493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:24.098483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d509412-e900-42ed-b4b8-2279c27f7747 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:24.098913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d509412-e900-42ed-b4b8-2279c27f7747 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:24.099053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d509412-e900-42ed-b4b8-2279c27f7747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:24.166656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1d509412-e900-42ed-b4b8-2279c27f7747 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:24.167895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1d509412-e900-42ed-b4b8-2279c27f7747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:24.188314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:24.265406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:24.557165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1b601-28e4-4a8a-8f1e-8b6daf864a3b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:24.558752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-44a1b601-28e4-4a8a-8f1e-8b6daf864a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:24.671333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1b601-28e4-4a8a-8f1e-8b6daf864a3b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:24.692199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:24.746885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df09c707-c137-4b64-9501-b756de03da01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:24.747494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c707-c137-4b64-9501-b756de03da01 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:24.747774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c707-c137-4b64-9501-b756de03da01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:24.748303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df09c707-c137-4b64-9501-b756de03da01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:24.749957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-df09c707-c137-4b64-9501-b756de03da01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:24.859991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09c707-c137-4b64-9501-b756de03da01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:24.861438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df09c707-c137-4b64-9501-b756de03da01 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:24.880441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 735/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:26.945834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6efae-2005-43d0-9b3c-fe61460a1002 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:26.946397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6efae-2005-43d0-9b3c-fe61460a1002 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:26.946397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6efae-2005-43d0-9b3c-fe61460a1002 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:26.947769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:26.953263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96250691-997e-4adc-ac1b-2a1ce04ca8a5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:26.953705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96250691-997e-4adc-ac1b-2a1ce04ca8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:26.953838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96250691-997e-4adc-ac1b-2a1ce04ca8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:27.030673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:27.033915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-96250691-997e-4adc-ac1b-2a1ce04ca8a5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:27.035919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-96250691-997e-4adc-ac1b-2a1ce04ca8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:27.140315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:27.437236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6efae-2005-43d0-9b3c-fe61460a1002 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:27.438838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5ac6efae-2005-43d0-9b3c-fe61460a1002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:27.546058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6efae-2005-43d0-9b3c-fe61460a1002 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:27.596458 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:27.623728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f5ae1-95ad-4c71-bcca-0a8085286cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:27.624333 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f5ae1-95ad-4c71-bcca-0a8085286cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:27.624609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f5ae1-95ad-4c71-bcca-0a8085286cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:27.624878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f5ae1-95ad-4c71-bcca-0a8085286cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:27.626508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-865f5ae1-95ad-4c71-bcca-0a8085286cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:27.741845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f5ae1-95ad-4c71-bcca-0a8085286cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:27.743149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f5ae1-95ad-4c71-bcca-0a8085286cc1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:27.768170 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 719/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:29.758518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c5baa-ec7b-4733-9a4b-ba1ef3e5aa66 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:29.759087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c5baa-ec7b-4733-9a4b-ba1ef3e5aa66 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:29.759087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c5baa-ec7b-4733-9a4b-ba1ef3e5aa66 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:29.760423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:29.765902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4009cb-7ed8-40c8-9192-40bac9fe5574 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:29.766334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4009cb-7ed8-40c8-9192-40bac9fe5574 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:29.766451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4009cb-7ed8-40c8-9192-40bac9fe5574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:29.836669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2e4009cb-7ed8-40c8-9192-40bac9fe5574 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:29.837784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2e4009cb-7ed8-40c8-9192-40bac9fe5574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:29.870573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:29.944607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:30.245570 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c5baa-ec7b-4733-9a4b-ba1ef3e5aa66 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:30.247132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-061c5baa-ec7b-4733-9a4b-ba1ef3e5aa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:30.353599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c5baa-ec7b-4733-9a4b-ba1ef3e5aa66 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:30.383264 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:30.428815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63140044-6cf7-4a06-9133-9d4a65cca7d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:30.429430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63140044-6cf7-4a06-9133-9d4a65cca7d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:30.429709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63140044-6cf7-4a06-9133-9d4a65cca7d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:30.430082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63140044-6cf7-4a06-9133-9d4a65cca7d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:30.431744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-63140044-6cf7-4a06-9133-9d4a65cca7d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:30.546473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63140044-6cf7-4a06-9133-9d4a65cca7d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:30.547926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63140044-6cf7-4a06-9133-9d4a65cca7d7 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:30.565893 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 738/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:32.564977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e1963-f4ff-4656-a744-2b81bd3151f7 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:32.565562 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e1963-f4ff-4656-a744-2b81bd3151f7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:32.565562 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e1963-f4ff-4656-a744-2b81bd3151f7 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:32.567601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:32.573050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e5e62-d0fd-4d75-aaf5-f2b2e2ebf7dd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:32.573506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e5e62-d0fd-4d75-aaf5-f2b2e2ebf7dd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:32.573615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e5e62-d0fd-4d75-aaf5-f2b2e2ebf7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:32.640812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0a5e5e62-d0fd-4d75-aaf5-f2b2e2ebf7dd None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:32.642063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0a5e5e62-d0fd-4d75-aaf5-f2b2e2ebf7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:32.651142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:32.749051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:33.034251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e1963-f4ff-4656-a744-2b81bd3151f7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:33.036933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c21e1963-f4ff-4656-a744-2b81bd3151f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:33.145679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21e1963-f4ff-4656-a744-2b81bd3151f7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:33.202610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:33.223354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b1eef-1e86-401e-83bd-08c4f09b71a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:33.224047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b1eef-1e86-401e-83bd-08c4f09b71a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:33.224335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b1eef-1e86-401e-83bd-08c4f09b71a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:33.224613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b1eef-1e86-401e-83bd-08c4f09b71a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:33.226280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d7b1eef-1e86-401e-83bd-08c4f09b71a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:33.341004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b1eef-1e86-401e-83bd-08c4f09b71a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:33.342375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b1eef-1e86-401e-83bd-08c4f09b71a8 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:33.370454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 722/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:35.388756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce343cce-110b-41de-a466-044ee4b80e4b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:35.389279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce343cce-110b-41de-a466-044ee4b80e4b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:35.389279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce343cce-110b-41de-a466-044ee4b80e4b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:35.390624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:35.396611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f4177-e888-4723-bd7f-fe762affc389 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:35.397052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f4177-e888-4723-bd7f-fe762affc389 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:35.397146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f4177-e888-4723-bd7f-fe762affc389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:35.465702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ec7f4177-e888-4723-bd7f-fe762affc389 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:35.466986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec7f4177-e888-4723-bd7f-fe762affc389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:35.507534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:35.569710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:02:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:35.860811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce343cce-110b-41de-a466-044ee4b80e4b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:35.862450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ce343cce-110b-41de-a466-044ee4b80e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:35.971457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce343cce-110b-41de-a466-044ee4b80e4b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:36.012804 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:36.050221 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11195640-d745-487d-a0e1-572e558e4e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:36.050915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11195640-d745-487d-a0e1-572e558e4e4e service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:36.051198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11195640-d745-487d-a0e1-572e558e4e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:36.051473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11195640-d745-487d-a0e1-572e558e4e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:36.053211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11195640-d745-487d-a0e1-572e558e4e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:36.167981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11195640-d745-487d-a0e1-572e558e4e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:36.169448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11195640-d745-487d-a0e1-572e558e4e4e service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:36.188720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 741/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:38.248870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a44665-9a27-40e0-a3e7-9eb9ff41de6a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:38.249442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a44665-9a27-40e0-a3e7-9eb9ff41de6a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:38.249442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a44665-9a27-40e0-a3e7-9eb9ff41de6a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:38.250779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:38.256825 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb87807-3f4f-48a3-b9da-e7dd7e362241 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:38.257265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb87807-3f4f-48a3-b9da-e7dd7e362241 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:38.257359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb87807-3f4f-48a3-b9da-e7dd7e362241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:38.325934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2cb87807-3f4f-48a3-b9da-e7dd7e362241 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:38.327165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2cb87807-3f4f-48a3-b9da-e7dd7e362241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:38.345709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:38.428962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:38.729408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a44665-9a27-40e0-a3e7-9eb9ff41de6a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:38.730996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-02a44665-9a27-40e0-a3e7-9eb9ff41de6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:38.845528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a44665-9a27-40e0-a3e7-9eb9ff41de6a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:38.915530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:38.923238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d57604-3aae-4a61-b341-41dd9c4966b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:38.923852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d57604-3aae-4a61-b341-41dd9c4966b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:38.924161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d57604-3aae-4a61-b341-41dd9c4966b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:38.924438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d57604-3aae-4a61-b341-41dd9c4966b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:38.926229 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-57d57604-3aae-4a61-b341-41dd9c4966b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:39.039028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d57604-3aae-4a61-b341-41dd9c4966b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:39.040485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d57604-3aae-4a61-b341-41dd9c4966b3 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:39.060640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 725/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:41.021183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c65e4-e448-46f2-9c62-dc84617df9ef None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:41.021789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c65e4-e448-46f2-9c62-dc84617df9ef None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:41.021789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c65e4-e448-46f2-9c62-dc84617df9ef None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:41.023162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:02:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:02:41.028712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ad6c7-5c63-4fe6-ae7f-77829981de18 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:41.029169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ad6c7-5c63-4fe6-ae7f-77829981de18 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:41.029269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ad6c7-5c63-4fe6-ae7f-77829981de18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:41.097058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-211ad6c7-5c63-4fe6-ae7f-77829981de18 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:02:41.098823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-211ad6c7-5c63-4fe6-ae7f-77829981de18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:41.153375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:41.204136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:02:41.487912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c65e4-e448-46f2-9c62-dc84617df9ef None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:02:41.489469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a94c65e4-e448-46f2-9c62-dc84617df9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:41.601466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c65e4-e448-46f2-9c62-dc84617df9ef None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:02:41.650500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:41.681753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d1ae0d-acb5-470b-b8c5-cfe2b9d661e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:02:41.682877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1ae0d-acb5-470b-b8c5-cfe2b9d661e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:02:41.683393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1ae0d-acb5-470b-b8c5-cfe2b9d661e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:02:41.683844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d1ae0d-acb5-470b-b8c5-cfe2b9d661e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:02:41.686301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-97d1ae0d-acb5-470b-b8c5-cfe2b9d661e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:02:41.807212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1ae0d-acb5-470b-b8c5-cfe2b9d661e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:02:41.808666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d1ae0d-acb5-470b-b8c5-cfe2b9d661e7 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:02:41.827452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 744/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:02:51.202385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=22 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:02:51.825101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=66 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:01.785837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e2ab9-affd-43da-ac10-12af5cd06695 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:01.787081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e2ab9-affd-43da-ac10-12af5cd06695 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:01.787081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e2ab9-affd-43da-ac10-12af5cd06695 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:01.787845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 727/1471] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:03:01.794073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee586c-4284-4569-b556-c63e3db5e23e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:01.794531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee586c-4284-4569-b556-c63e3db5e23e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:01.794640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee586c-4284-4569-b556-c63e3db5e23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:01.884497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b6ee586c-4284-4569-b556-c63e3db5e23e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:03:01.886011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b6ee586c-4284-4569-b556-c63e3db5e23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:02.015167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 745/1472] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:03:02.351868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3d88e-5594-4fe8-a1a9-0ad9791b1eff None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:02.352157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3d88e-5594-4fe8-a1a9-0ad9791b1eff None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:02.352434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3d88e-5594-4fe8-a1a9-0ad9791b1eff None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:02.353787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 728/1473] 10.222.0.16 () {54 vars in 967 bytes} [Fri Aug 19 21:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:03:02.358935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54395015-6350-481f-8fe2-090ff4b27331 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:02.359227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54395015-6350-481f-8fe2-090ff4b27331 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:02.359518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54395015-6350-481f-8fe2-090ff4b27331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:02.438586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-54395015-6350-481f-8fe2-090ff4b27331 None None] MFA Rules not processed for user `b957b367ddf2483f84056ccec57fc515`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:03:02.439802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-54395015-6350-481f-8fe2-090ff4b27331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:02.461477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:02.546221 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 746/1474] 10.222.0.16 () {58 vars in 1064 bytes} [Fri Aug 19 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:03:02.554391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3d88e-5594-4fe8-a1a9-0ad9791b1eff None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:03:02.555935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f2c3d88e-5594-4fe8-a1a9-0ad9791b1eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:02.672544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3d88e-5594-4fe8-a1a9-0ad9791b1eff None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:03:02.749951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3713e475-3a80-4717-92cf-98fe81cc897c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b957b367ddf2483f84056ccec57fc515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:03:02.750559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713e475-3a80-4717-92cf-98fe81cc897c service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:02.750844 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713e475-3a80-4717-92cf-98fe81cc897c service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:02.751119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3713e475-3a80-4717-92cf-98fe81cc897c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:02.752759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3713e475-3a80-4717-92cf-98fe81cc897c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:02.810069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:02.868566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713e475-3a80-4717-92cf-98fe81cc897c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:03:02.870035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3713e475-3a80-4717-92cf-98fe81cc897c service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:03:02.889167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 729/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:03:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:03:04.972012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928674a3-36aa-4909-bc9c-6c36e92535d4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:04.972568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928674a3-36aa-4909-bc9c-6c36e92535d4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:04.972568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928674a3-36aa-4909-bc9c-6c36e92535d4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:04.974104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 747/1476] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:03:04.979277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406d934-ff9c-41de-91c6-5f7a5931c272 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:04.979687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406d934-ff9c-41de-91c6-5f7a5931c272 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:04.979832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0406d934-ff9c-41de-91c6-5f7a5931c272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:05.046849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0406d934-ff9c-41de-91c6-5f7a5931c272 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:03:05.047991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0406d934-ff9c-41de-91c6-5f7a5931c272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:05.148375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 730/1477] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:03:05.424269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928674a3-36aa-4909-bc9c-6c36e92535d4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:03:05.425724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-928674a3-36aa-4909-bc9c-6c36e92535d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:05.551947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928674a3-36aa-4909-bc9c-6c36e92535d4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:03:05.600441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:05.649028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df3f48-0536-4b2a-92e7-c0d8a5745441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:03:05.649646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df3f48-0536-4b2a-92e7-c0d8a5745441 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:05.649962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df3f48-0536-4b2a-92e7-c0d8a5745441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:05.650237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df3f48-0536-4b2a-92e7-c0d8a5745441 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:05.650884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df3f48-0536-4b2a-92e7-c0d8a5745441 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:03:05.652688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-94df3f48-0536-4b2a-92e7-c0d8a5745441 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:03:05.652688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 19 21:03:05.652688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: Aug 19 21:03:05.653934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df3f48-0536-4b2a-92e7-c0d8a5745441 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:03:05.684029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 748/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Fri Aug 19 21:03:05 2022] GET /identity/v3/projects => generated 2553 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:03:07.657177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2efa4-78ff-448b-831e-df96edc08ab2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:07.657715 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2efa4-78ff-448b-831e-df96edc08ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:07.657984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2efa4-78ff-448b-831e-df96edc08ab2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:07.659114 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 731/1479] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:03:07.665279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbcad6-4f21-48a2-bb1d-86490c232be3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:07.665701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbcad6-4f21-48a2-bb1d-86490c232be3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:07.665869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fbcad6-4f21-48a2-bb1d-86490c232be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:07.853658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-85fbcad6-4f21-48a2-bb1d-86490c232be3 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:03:07.854913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-85fbcad6-4f21-48a2-bb1d-86490c232be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:07.950534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 749/1480] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:03:08.251791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2efa4-78ff-448b-831e-df96edc08ab2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:03:08.253401 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-65a2efa4-78ff-448b-831e-df96edc08ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:08.263262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:08.399902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2efa4-78ff-448b-831e-df96edc08ab2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:03:08.479712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:08.515900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8349dd-eab3-4900-921f-5d7afa2c2484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:03:08.516508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8349dd-eab3-4900-921f-5d7afa2c2484 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:08.516782 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8349dd-eab3-4900-921f-5d7afa2c2484 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:08.517052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8349dd-eab3-4900-921f-5d7afa2c2484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:08.518679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dc8349dd-eab3-4900-921f-5d7afa2c2484 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:08.606471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8349dd-eab3-4900-921f-5d7afa2c2484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:03:08.608076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8349dd-eab3-4900-921f-5d7afa2c2484 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:03:08.626621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 732/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:03:10.753541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01469132-8ff9-4040-9c48-9273c589246c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:10.754300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01469132-8ff9-4040-9c48-9273c589246c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:10.754300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01469132-8ff9-4040-9c48-9273c589246c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:10.755633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 750/1482] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:03:10.762032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f478af-2b4f-4595-b5b1-79139cfd1c04 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:10.762543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f478af-2b4f-4595-b5b1-79139cfd1c04 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:10.762647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f478af-2b4f-4595-b5b1-79139cfd1c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:10.831197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-55f478af-2b4f-4595-b5b1-79139cfd1c04 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:03:10.832433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-55f478af-2b4f-4595-b5b1-79139cfd1c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:10.886389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:10.938466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 733/1483] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:03:11.226515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01469132-8ff9-4040-9c48-9273c589246c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:03:11.228199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01469132-8ff9-4040-9c48-9273c589246c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:11.253115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:11.335310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01469132-8ff9-4040-9c48-9273c589246c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:03:11.410167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e6f78-186b-4981-a23c-b35938010e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:03:11.410694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e6f78-186b-4981-a23c-b35938010e83 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:11.410972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e6f78-186b-4981-a23c-b35938010e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:11.411244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e6f78-186b-4981-a23c-b35938010e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:11.412839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c30e6f78-186b-4981-a23c-b35938010e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:11.486858 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:11.524720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e6f78-186b-4981-a23c-b35938010e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:03:11.526191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e6f78-186b-4981-a23c-b35938010e83 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:03:11.543881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 751/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:03:13.564195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a893dc0-c5af-4afb-922b-1d63aa92f7c9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:13.564760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a893dc0-c5af-4afb-922b-1d63aa92f7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:13.564760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a893dc0-c5af-4afb-922b-1d63aa92f7c9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:13.566185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 734/1485] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:03:13.571980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee331b-85b8-4041-994a-2be1508a6e77 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:13.572443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee331b-85b8-4041-994a-2be1508a6e77 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:13.572555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eee331b-85b8-4041-994a-2be1508a6e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:13.639110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8eee331b-85b8-4041-994a-2be1508a6e77 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:03:13.640332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8eee331b-85b8-4041-994a-2be1508a6e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:13.741824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 752/1486] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:03:14.041534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a893dc0-c5af-4afb-922b-1d63aa92f7c9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:03:14.043218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4a893dc0-c5af-4afb-922b-1d63aa92f7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:14.150080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a893dc0-c5af-4afb-922b-1d63aa92f7c9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:03:14.227982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:14.229972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41479d-44bf-4594-8b82-095b9895829c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:03:14.230649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41479d-44bf-4594-8b82-095b9895829c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:14.231196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41479d-44bf-4594-8b82-095b9895829c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:14.231644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41479d-44bf-4594-8b82-095b9895829c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:14.233477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9d41479d-44bf-4594-8b82-095b9895829c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:14.346017 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41479d-44bf-4594-8b82-095b9895829c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:03:14.347399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41479d-44bf-4594-8b82-095b9895829c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:03:14.368045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 735/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:03:16.501423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f18fba-352a-4733-a7f3-dab113465d41 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:16.502041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f18fba-352a-4733-a7f3-dab113465d41 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:16.502041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f18fba-352a-4733-a7f3-dab113465d41 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:16.503358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 753/1488] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:03:16.509595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfda71e-5f4c-4d8d-b02a-8f7e6da4607d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:16.510103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfda71e-5f4c-4d8d-b02a-8f7e6da4607d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:16.510196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfda71e-5f4c-4d8d-b02a-8f7e6da4607d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:16.583238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8dfda71e-5f4c-4d8d-b02a-8f7e6da4607d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:03:16.584465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8dfda71e-5f4c-4d8d-b02a-8f7e6da4607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:16.645692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:16.698234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 736/1489] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:03:16.999892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f18fba-352a-4733-a7f3-dab113465d41 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:03:17.001451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-89f18fba-352a-4733-a7f3-dab113465d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:17.039738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:17.111448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f18fba-352a-4733-a7f3-dab113465d41 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:03:17.188496 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4518661f-9e7b-4541-91fb-72bd14d2d6b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:03:17.189097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518661f-9e7b-4541-91fb-72bd14d2d6b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:17.189365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518661f-9e7b-4541-91fb-72bd14d2d6b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:17.189626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4518661f-9e7b-4541-91fb-72bd14d2d6b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:17.191249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4518661f-9e7b-4541-91fb-72bd14d2d6b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:17.290123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:17.324461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518661f-9e7b-4541-91fb-72bd14d2d6b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:03:17.325943 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4518661f-9e7b-4541-91fb-72bd14d2d6b8 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:03:17.346263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 754/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:03:19.478162 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca74eab-2f06-4784-aa5e-51486e435249 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:19.478741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca74eab-2f06-4784-aa5e-51486e435249 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:19.478741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca74eab-2f06-4784-aa5e-51486e435249 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:19.480062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 737/1491] 10.222.0.16 () {54 vars in 927 bytes} [Fri Aug 19 21:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:03:19.485758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8675-6a24-4ea0-b336-d763ae543415 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:19.486284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8675-6a24-4ea0-b336-d763ae543415 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:19.486392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea8675-6a24-4ea0-b336-d763ae543415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:19.556640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9fea8675-6a24-4ea0-b336-d763ae543415 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:03:19.557904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9fea8675-6a24-4ea0-b336-d763ae543415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:19.656429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 755/1492] 10.222.0.16 () {58 vars in 1024 bytes} [Fri Aug 19 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:03:19.953237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca74eab-2f06-4784-aa5e-51486e435249 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:03:19.954814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dca74eab-2f06-4784-aa5e-51486e435249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:20.066723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca74eab-2f06-4784-aa5e-51486e435249 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:03:20.143930 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25268e54-53ba-42e7-b1c4-c000f21b3c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:03:20.144543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25268e54-53ba-42e7-b1c4-c000f21b3c78 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:03:20.144824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25268e54-53ba-42e7-b1c4-c000f21b3c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:03:20.145100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25268e54-53ba-42e7-b1c4-c000f21b3c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:03:20.146768 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-25268e54-53ba-42e7-b1c4-c000f21b3c78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:03:20.156901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:20.262677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25268e54-53ba-42e7-b1c4-c000f21b3c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:03:20.264128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25268e54-53ba-42e7-b1c4-c000f21b3c78 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:03:20.282810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 738/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:03:29.654641 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=82 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:03:30.280447 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=50 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:32.868110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7417d-9e20-4a87-a867-2cf6ac355e91 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:32.868710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7417d-9e20-4a87-a867-2cf6ac355e91 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:32.868710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7417d-9e20-4a87-a867-2cf6ac355e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:32.950218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3bc7417d-9e20-4a87-a867-2cf6ac355e91 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:04:32.951448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3bc7417d-9e20-4a87-a867-2cf6ac355e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:33.060673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 756/1494] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:04:33.069297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e85436-109a-496a-a358-6ed588d07d70 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:33.070157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e85436-109a-496a-a358-6ed588d07d70 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:33.070157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e85436-109a-496a-a358-6ed588d07d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:33.149946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-48e85436-109a-496a-a358-6ed588d07d70 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:04:33.151095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-48e85436-109a-496a-a358-6ed588d07d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:33.247423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 739/1495] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:04:33.252418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7417d-9e20-4a87-a867-2cf6ac355e91 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:04:33.253621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3bc7417d-9e20-4a87-a867-2cf6ac355e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:33.265168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7417d-9e20-4a87-a867-2cf6ac355e91 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:04:33.348731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:33.393425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:04:33.394069 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:33.394350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:33.394623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:33.395268 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:04:33.395571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:04:33.396765 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:04:33.396765 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 19 21:04:33.396765 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: Aug 19 21:04:33.397674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:04:33.409899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 757/1496] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:04:33.417976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e85436-109a-496a-a358-6ed588d07d70 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:04:33.428083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e85436-109a-496a-a358-6ed588d07d70 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:04:33.488907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:33.524637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b2fad-b168-4ab9-8fae-f628aa044469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:04:33.525237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b2fad-b168-4ab9-8fae-f628aa044469 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:33.525514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b2fad-b168-4ab9-8fae-f628aa044469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:33.525807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b2fad-b168-4ab9-8fae-f628aa044469 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:33.526428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b2fad-b168-4ab9-8fae-f628aa044469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:04:33.528145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-091b2fad-b168-4ab9-8fae-f628aa044469 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:04:33.528145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 19 21:04:33.528145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: Aug 19 21:04:33.529342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b2fad-b168-4ab9-8fae-f628aa044469 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:04:33.586699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 740/1497] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:04:33 2022] POST /identity/v3/projects => generated 379 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:04:33.590866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:04:33.592460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:33.631589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca6419-38d1-4c5b-a8e7-4d711a12cd4e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:04:33.745288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab065cc-9526-47ae-b7b2-adb27b47f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:04:33.745834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab065cc-9526-47ae-b7b2-adb27b47f72a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:33.746111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab065cc-9526-47ae-b7b2-adb27b47f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:33.746382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab065cc-9526-47ae-b7b2-adb27b47f72a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:33.747011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab065cc-9526-47ae-b7b2-adb27b47f72a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:04:33.748011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-6ab065cc-9526-47ae-b7b2-adb27b47f72a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:04:33.748011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 19 21:04:33.748011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: Aug 19 21:04:33.749222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab065cc-9526-47ae-b7b2-adb27b47f72a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:04:33.792235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 758/1498] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:04:33 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:04:33.796958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b2fad-b168-4ab9-8fae-f628aa044469 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:04:33.797914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-091b2fad-b168-4ab9-8fae-f628aa044469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:33.861299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b2fad-b168-4ab9-8fae-f628aa044469 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:04:33.947510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:33.959410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e84ae-7bf6-4a82-9c37-2450deb3a83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:04:33.960029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e84ae-7bf6-4a82-9c37-2450deb3a83e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:33.960305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e84ae-7bf6-4a82-9c37-2450deb3a83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:33.960578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e84ae-7bf6-4a82-9c37-2450deb3a83e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:33.961812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e84ae-7bf6-4a82-9c37-2450deb3a83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:04:33.962287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-dd8e84ae-7bf6-4a82-9c37-2450deb3a83e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:04:33.962287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 19 21:04:33.962287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: Aug 19 21:04:33.963191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e84ae-7bf6-4a82-9c37-2450deb3a83e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:04:33.975633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 741/1499] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:04:33.979688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab065cc-9526-47ae-b7b2-adb27b47f72a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:04:33.981204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6ab065cc-9526-47ae-b7b2-adb27b47f72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:33.993295 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab065cc-9526-47ae-b7b2-adb27b47f72a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:04:34.019313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:34.095610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3ad4d-086c-45c3-87d3-b26caf44ec3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:04:34.096226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3ad4d-086c-45c3-87d3-b26caf44ec3e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:34.096508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3ad4d-086c-45c3-87d3-b26caf44ec3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:34.096777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3ad4d-086c-45c3-87d3-b26caf44ec3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:34.097418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3ad4d-086c-45c3-87d3-b26caf44ec3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:04:34.098465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-eec3ad4d-086c-45c3-87d3-b26caf44ec3e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:04:34.098465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 19 21:04:34.098465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: Aug 19 21:04:34.099527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3ad4d-086c-45c3-87d3-b26caf44ec3e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:04:34.111891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 759/1500] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:04:34.116897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e84ae-7bf6-4a82-9c37-2450deb3a83e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:04:34.118494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd8e84ae-7bf6-4a82-9c37-2450deb3a83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:34.152543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e84ae-7bf6-4a82-9c37-2450deb3a83e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:04:34.246745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1a68b-24c4-4341-baf8-99bdc215fbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:04:34.247402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1a68b-24c4-4341-baf8-99bdc215fbf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:34.247676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1a68b-24c4-4341-baf8-99bdc215fbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:34.247946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1a68b-24c4-4341-baf8-99bdc215fbf3 admin admin] PATH_INFO: `/v3/projects/1e641709ec1d4750a4e2eb947f4cba81/users/7d3919b2ffab4b00b532ebcf3375218c/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:34.271790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1a68b-24c4-4341-baf8-99bdc215fbf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e641709ec1d4750a4e2eb947f4cba81, user_id=7d3919b2ffab4b00b532ebcf3375218c, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:04:34.272767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-b7f1a68b-24c4-4341-baf8-99bdc215fbf3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:04:34.272767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 19 21:04:34.272767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: Aug 19 21:04:34.274764 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1a68b-24c4-4341-baf8-99bdc215fbf3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:04:34.299811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 742/1501] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:04:34 2022] PUT /identity/v3/projects/1e641709ec1d4750a4e2eb947f4cba81/users/7d3919b2ffab4b00b532ebcf3375218c/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:04:34.312478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a195a79-2b66-435f-a413-0eedc5cc1516 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:34.312769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a195a79-2b66-435f-a413-0eedc5cc1516 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:34.313064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a195a79-2b66-435f-a413-0eedc5cc1516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:34.370059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6a195a79-2b66-435f-a413-0eedc5cc1516 None None] MFA Rules not processed for user `7d3919b2ffab4b00b532ebcf3375218c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:04:34.371269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6a195a79-2b66-435f-a413-0eedc5cc1516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:34.450125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 760/1502] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:04:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:04:34.463095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9742aa-e6c5-4af1-a319-a68ea21079cb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:34.463379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9742aa-e6c5-4af1-a319-a68ea21079cb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:34.463653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9742aa-e6c5-4af1-a319-a68ea21079cb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:34.465043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 743/1503] 10.222.0.16 () {54 vars in 964 bytes} [Fri Aug 19 21:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:04:34.469718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fbaeb-638a-4ee1-81ff-80cb45af4cbe None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:34.470024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fbaeb-638a-4ee1-81ff-80cb45af4cbe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:34.470301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fbaeb-638a-4ee1-81ff-80cb45af4cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:34.503065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:34.548096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-808fbaeb-638a-4ee1-81ff-80cb45af4cbe None None] MFA Rules not processed for user `54d1ae400b5c4ae0955047b9e7750103`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:04:34.549208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-808fbaeb-638a-4ee1-81ff-80cb45af4cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:34.625629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 761/1504] 10.222.0.16 () {58 vars in 1061 bytes} [Fri Aug 19 21:04:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:04:34.633449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9742aa-e6c5-4af1-a319-a68ea21079cb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:04:34.635045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-de9742aa-e6c5-4af1-a319-a68ea21079cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:34.685227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:34.740358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9742aa-e6c5-4af1-a319-a68ea21079cb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:04:34.816270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfa97d-161c-423f-9988-6ab8a532e911 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:04:34.816815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfa97d-161c-423f-9988-6ab8a532e911 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:34.817098 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfa97d-161c-423f-9988-6ab8a532e911 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:34.817365 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edfa97d-161c-423f-9988-6ab8a532e911 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:34.819523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0edfa97d-161c-423f-9988-6ab8a532e911 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:34.921636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfa97d-161c-423f-9988-6ab8a532e911 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:04:34.923091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edfa97d-161c-423f-9988-6ab8a532e911 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:04:34.926511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:34.941243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 744/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:04:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:04:34.987252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86a4ed-f494-4ef1-bf1f-4c60ad939b79 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:34.987660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86a4ed-f494-4ef1-bf1f-4c60ad939b79 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:34.987813 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86a4ed-f494-4ef1-bf1f-4c60ad939b79 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:34.989119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 762/1506] 10.222.0.16 () {52 vars in 830 bytes} [Fri Aug 19 21:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:04:34.993661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfa97d-161c-423f-9988-6ab8a532e911 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:04:34.994912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0edfa97d-161c-423f-9988-6ab8a532e911 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:35.098436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edfa97d-161c-423f-9988-6ab8a532e911 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:04:35.192787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:04:35.193449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:35.193727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:35.194028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] PATH_INFO: `/v3/users/7d3919b2ffab4b00b532ebcf3375218c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:35.206088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d3919b2ffab4b00b532ebcf3375218c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:04:35.207099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:04:35.207099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 19 21:04:35.207099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: Aug 19 21:04:35.208469 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:04:35.218514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:35.275356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d3919b2ffab4b00b532ebcf3375218c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:04:35.284716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d3919b2ffab4b00b532ebcf3375218c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:04:35.285206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d3919b2ffab4b00b532ebcf3375218c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:04:35.285704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d3919b2ffab4b00b532ebcf3375218c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:04:35.301612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d3919b2ffab4b00b532ebcf3375218c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:04:35.330229 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 745/1507] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:04:34 2022] DELETE /identity/v3/users/7d3919b2ffab4b00b532ebcf3375218c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:04:35.334675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f5d32-0573-49bc-80a0-e42b0464b966 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:35.334990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f5d32-0573-49bc-80a0-e42b0464b966 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:35.335292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018f5d32-0573-49bc-80a0-e42b0464b966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:35.393433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-018f5d32-0573-49bc-80a0-e42b0464b966 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:04:35.394589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-018f5d32-0573-49bc-80a0-e42b0464b966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:35.471107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:35.484631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 763/1508] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:04:35.494817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:04:35.496071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:35.595485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af7eb8-2512-41fe-960e-e63ede2c7fd0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:04:35.654532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:35.679189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf7b207-2120-406a-bc5c-f92125d8f6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:04:35.679818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b207-2120-406a-bc5c-f92125d8f6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:35.680101 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b207-2120-406a-bc5c-f92125d8f6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:35.680383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf7b207-2120-406a-bc5c-f92125d8f6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:35.682143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fdf7b207-2120-406a-bc5c-f92125d8f6c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:35.795794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7b207-2120-406a-bc5c-f92125d8f6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:04:35.797234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf7b207-2120-406a-bc5c-f92125d8f6c8 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:04:35.815663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 746/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:04:36.150371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f5d32-0573-49bc-80a0-e42b0464b966 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:04:36.151719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-018f5d32-0573-49bc-80a0-e42b0464b966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:04:36.164359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018f5d32-0573-49bc-80a0-e42b0464b966 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:04:36.269130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:04:36.269778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:04:36.270065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:04:36.270336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] PATH_INFO: `/v3/projects/1e641709ec1d4750a4e2eb947f4cba81` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:04:36.280712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e641709ec1d4750a4e2eb947f4cba81)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:04:36.281731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:04:36.281731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 19 21:04:36.281731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: Aug 19 21:04:36.282986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:04:36.346497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e641709ec1d4750a4e2eb947f4cba81', 'request_id': 'req-4f09fff8-4598-4759-b2e7-11270903c7ca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:04:36.355612 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e641709ec1d4750a4e2eb947f4cba81', 'request_id': 'req-4f09fff8-4598-4759-b2e7-11270903c7ca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:04:36.361725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e641709ec1d4750a4e2eb947f4cba81', 'request_id': 'req-4f09fff8-4598-4759-b2e7-11270903c7ca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:04:36.391449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] The token cache is being invalidate because project 1e641709ec1d4750a4e2eb947f4cba81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:04:36.391737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:04:36.392269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4f09fff8-4598-4759-b2e7-11270903c7ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:04:36.393585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 764/1510] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:04:36 2022] DELETE /identity/v3/projects/1e641709ec1d4750a4e2eb947f4cba81 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:04:45.813377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=62 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:04:46.389373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:06:04.315907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bdaeb-f0b8-41f3-be87-6916b8228cd0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:04.316535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bdaeb-f0b8-41f3-be87-6916b8228cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:04.316903 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43bdaeb-f0b8-41f3-be87-6916b8228cd0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:04.318505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:06:04.324566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994a1d4-d246-425c-81fd-6bd619d02ec9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:04.325215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994a1d4-d246-425c-81fd-6bd619d02ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:04.325215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994a1d4-d246-425c-81fd-6bd619d02ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:04.424733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8994a1d4-d246-425c-81fd-6bd619d02ec9 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:06:04.426015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8994a1d4-d246-425c-81fd-6bd619d02ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:04.533005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:06:04.692890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bdaeb-f0b8-41f3-be87-6916b8228cd0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:06:04.695078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b43bdaeb-f0b8-41f3-be87-6916b8228cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:04.817815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43bdaeb-f0b8-41f3-be87-6916b8228cd0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:06:04.824265 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77669f14-96fd-4a0c-aaca-8ef9ff7c8b2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:06:04.824862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669f14-96fd-4a0c-aaca-8ef9ff7c8b2c service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:04.825141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669f14-96fd-4a0c-aaca-8ef9ff7c8b2c service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:04.825417 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669f14-96fd-4a0c-aaca-8ef9ff7c8b2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:04.833673 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77669f14-96fd-4a0c-aaca-8ef9ff7c8b2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:06:04.835326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77669f14-96fd-4a0c-aaca-8ef9ff7c8b2c service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:06:04.844843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 748/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:06:05.699107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2095a-76b2-4602-8745-5403c82e0be2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:05.699685 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2095a-76b2-4602-8745-5403c82e0be2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:05.699980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2095a-76b2-4602-8745-5403c82e0be2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:05.701646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 766/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:06:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:06:05.706800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb15831-02e3-4b77-abda-10e7a933daec None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:05.707251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb15831-02e3-4b77-abda-10e7a933daec None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:05.707347 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb15831-02e3-4b77-abda-10e7a933daec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:05.730682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cbb15831-02e3-4b77-abda-10e7a933daec None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:06:05.731952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cbb15831-02e3-4b77-abda-10e7a933daec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:05.740259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 749/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:06:05.896768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2095a-76b2-4602-8745-5403c82e0be2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:06:05.912680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2095a-76b2-4602-8745-5403c82e0be2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:06:05.919660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb3574-a8f9-4ae8-9607-5d07f0e4624e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:06:05.920280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb3574-a8f9-4ae8-9607-5d07f0e4624e service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:05.920557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb3574-a8f9-4ae8-9607-5d07f0e4624e service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:05.920829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb3574-a8f9-4ae8-9607-5d07f0e4624e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:05.930324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb3574-a8f9-4ae8-9607-5d07f0e4624e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:06:05.931969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb3574-a8f9-4ae8-9607-5d07f0e4624e service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:06:05.943444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 767/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:06:06.517637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e7e60-8af6-435e-8b16-a83d1d10ea2c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:06.518066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e7e60-8af6-435e-8b16-a83d1d10ea2c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:06.518470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e7e60-8af6-435e-8b16-a83d1d10ea2c None None] PATH_INFO: `/v3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:06.519807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 750/1517] 10.222.0.45 () {54 vars in 925 bytes} [Fri Aug 19 21:06:06 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:06:06.526514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229dcc6-3f7c-4652-9a95-0003f4c8b8da None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:06.526795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229dcc6-3f7c-4652-9a95-0003f4c8b8da None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:06.527069 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229dcc6-3f7c-4652-9a95-0003f4c8b8da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:06.587752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3229dcc6-3f7c-4652-9a95-0003f4c8b8da None None] MFA Rules not processed for user `824165e55d2f4742aa435f1edff9b403`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:06:06.588973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3229dcc6-3f7c-4652-9a95-0003f4c8b8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:06.605997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:06:06.661700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 768/1518] 10.222.0.45 () {58 vars in 1015 bytes} [Fri Aug 19 21:06:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:06:10.637870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9727566-38ce-40de-a5a2-b64933c724ff None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:10.638416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9727566-38ce-40de-a5a2-b64933c724ff None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:10.638416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9727566-38ce-40de-a5a2-b64933c724ff None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:10.639676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 751/1519] 10.222.0.16 () {54 vars in 961 bytes} [Fri Aug 19 21:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:06:10.644804 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9039cc-271f-4bbc-a6b7-24d52839c249 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:10.645210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9039cc-271f-4bbc-a6b7-24d52839c249 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:10.645354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9039cc-271f-4bbc-a6b7-24d52839c249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:10.667167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-df9039cc-271f-4bbc-a6b7-24d52839c249 None None] MFA Rules not processed for user `824165e55d2f4742aa435f1edff9b403`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:06:10.668383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-df9039cc-271f-4bbc-a6b7-24d52839c249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:10.677413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 769/1520] 10.222.0.16 () {58 vars in 1058 bytes} [Fri Aug 19 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:06:10.685098 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9727566-38ce-40de-a5a2-b64933c724ff None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:06:10.698486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9727566-38ce-40de-a5a2-b64933c724ff None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:06:10.703530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a490c6-41be-4a8b-bf8f-d6ddf8b4935a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:06:10.704125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a490c6-41be-4a8b-bf8f-d6ddf8b4935a service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:10.704397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a490c6-41be-4a8b-bf8f-d6ddf8b4935a service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:10.704666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a490c6-41be-4a8b-bf8f-d6ddf8b4935a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:10.715397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a490c6-41be-4a8b-bf8f-d6ddf8b4935a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:06:10.716990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a490c6-41be-4a8b-bf8f-d6ddf8b4935a service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:06:10.729507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 752/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:06:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:06:17.169184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d1fe-1b43-4350-970c-8f35c3b7fa6f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:17.169805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d1fe-1b43-4350-970c-8f35c3b7fa6f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:17.169805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7d1fe-1b43-4350-970c-8f35c3b7fa6f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:17.171114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:06:17.175995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d2adc-93eb-472f-ab93-2db4002d7f15 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:17.176402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d2adc-93eb-472f-ab93-2db4002d7f15 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:17.176584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d2adc-93eb-472f-ab93-2db4002d7f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:17.196365 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-239d2adc-93eb-472f-ab93-2db4002d7f15 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:06:17.198588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-239d2adc-93eb-472f-ab93-2db4002d7f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:17.206398 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:06:17.367247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d1fe-1b43-4350-970c-8f35c3b7fa6f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:06:17.383301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7d1fe-1b43-4350-970c-8f35c3b7fa6f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:06:17.389511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e0e77-4de3-4ec6-b085-af61b95d91c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:06:17.390147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e0e77-4de3-4ec6-b085-af61b95d91c8 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:17.390421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e0e77-4de3-4ec6-b085-af61b95d91c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:17.390691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e0e77-4de3-4ec6-b085-af61b95d91c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:17.399314 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e0e77-4de3-4ec6-b085-af61b95d91c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:06:17.401210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e0e77-4de3-4ec6-b085-af61b95d91c8 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:06:17.411182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 771/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:06:18.246238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a074e-fa79-4c44-8986-2379f89fde40 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:18.246798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a074e-fa79-4c44-8986-2379f89fde40 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:18.246798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a074e-fa79-4c44-8986-2379f89fde40 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:18.248108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Aug 19 21:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:06:18.253187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee029f-68a7-4baa-a242-6de6fc7cdbdf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:18.253617 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee029f-68a7-4baa-a242-6de6fc7cdbdf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:18.253866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee029f-68a7-4baa-a242-6de6fc7cdbdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:18.274346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6cee029f-68a7-4baa-a242-6de6fc7cdbdf None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:06:18.276292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6cee029f-68a7-4baa-a242-6de6fc7cdbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:18.286000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Aug 19 21:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:06:18.441026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a074e-fa79-4c44-8986-2379f89fde40 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:06:18.456154 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a074e-fa79-4c44-8986-2379f89fde40 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:06:18.461592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b92277a-079a-4fe6-971c-dbcc2fc10ea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:06:18.462244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92277a-079a-4fe6-971c-dbcc2fc10ea8 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:18.462530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92277a-079a-4fe6-971c-dbcc2fc10ea8 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:18.463496 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b92277a-079a-4fe6-971c-dbcc2fc10ea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:18.471794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92277a-079a-4fe6-971c-dbcc2fc10ea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:06:18.473383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b92277a-079a-4fe6-971c-dbcc2fc10ea8 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:06:18.483361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 755/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:06:22.310465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc28e6d-4e25-4d88-a4b6-545a13e61f24 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:22.311047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc28e6d-4e25-4d88-a4b6-545a13e61f24 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:22.311047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc28e6d-4e25-4d88-a4b6-545a13e61f24 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:22.312367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:06:22.317667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1862ec2-5b94-4188-afe6-9052e28f9893 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:22.318165 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1862ec2-5b94-4188-afe6-9052e28f9893 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:22.318274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1862ec2-5b94-4188-afe6-9052e28f9893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:22.339492 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b1862ec2-5b94-4188-afe6-9052e28f9893 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:06:22.340623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b1862ec2-5b94-4188-afe6-9052e28f9893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:22.350514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:06:22.641296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc28e6d-4e25-4d88-a4b6-545a13e61f24 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:06:22.643023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-efc28e6d-4e25-4d88-a4b6-545a13e61f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:22.722614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc28e6d-4e25-4d88-a4b6-545a13e61f24 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:06:22.726182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee3b8f-4831-44fb-b041-30e6933ac1b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:06:22.726788 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee3b8f-4831-44fb-b041-30e6933ac1b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:22.727061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee3b8f-4831-44fb-b041-30e6933ac1b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:22.727328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee3b8f-4831-44fb-b041-30e6933ac1b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:22.737467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee3b8f-4831-44fb-b041-30e6933ac1b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:06:22.738933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee3b8f-4831-44fb-b041-30e6933ac1b5 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:06:22.748400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 774/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:06:24.649486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4f01c-803e-40ef-870f-b36d77b464f2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:24.650138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4f01c-803e-40ef-870f-b36d77b464f2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:24.650138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4f01c-803e-40ef-870f-b36d77b464f2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:24.651790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:06:24.657152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780493b9-8153-4c48-be1e-bb42b7b96a3e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:24.657609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780493b9-8153-4c48-be1e-bb42b7b96a3e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:24.657715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780493b9-8153-4c48-be1e-bb42b7b96a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:24.678024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-780493b9-8153-4c48-be1e-bb42b7b96a3e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:06:24.678914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-780493b9-8153-4c48-be1e-bb42b7b96a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:24.687528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:06:24.976224 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4f01c-803e-40ef-870f-b36d77b464f2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:06:24.987747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4f01c-803e-40ef-870f-b36d77b464f2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:06:24.991323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239ea02-b189-4a55-87fa-da996190a362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:06:24.991913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239ea02-b189-4a55-87fa-da996190a362 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:24.992194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239ea02-b189-4a55-87fa-da996190a362 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:24.992469 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239ea02-b189-4a55-87fa-da996190a362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:25.001628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239ea02-b189-4a55-87fa-da996190a362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:06:25.003091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239ea02-b189-4a55-87fa-da996190a362 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:06:25.012533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 758/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:06:28.481506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d6d51-42c0-4f06-bf3d-31f53e3953d3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:28.482064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d6d51-42c0-4f06-bf3d-31f53e3953d3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:28.482064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d6d51-42c0-4f06-bf3d-31f53e3953d3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:28.483419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:06:28.488561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5fa57-ef0d-44e9-8026-65f83e9a973e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:28.489010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5fa57-ef0d-44e9-8026-65f83e9a973e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:28.489141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5fa57-ef0d-44e9-8026-65f83e9a973e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:28.497288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:06:28.510473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d9c5fa57-ef0d-44e9-8026-65f83e9a973e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:06:28.511680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d9c5fa57-ef0d-44e9-8026-65f83e9a973e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:28.519519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:06:29.077294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d6d51-42c0-4f06-bf3d-31f53e3953d3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:06:29.078989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eb8d6d51-42c0-4f06-bf3d-31f53e3953d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:29.147330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:06:29.161588 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8d6d51-42c0-4f06-bf3d-31f53e3953d3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:06:29.165131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc52f0b-ee21-4426-b1dd-2d024972b6a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:06:29.166836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc52f0b-ee21-4426-b1dd-2d024972b6a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:29.167109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc52f0b-ee21-4426-b1dd-2d024972b6a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:29.167379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc52f0b-ee21-4426-b1dd-2d024972b6a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:29.176350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc52f0b-ee21-4426-b1dd-2d024972b6a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:06:29.177838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc52f0b-ee21-4426-b1dd-2d024972b6a2 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:06:29.188016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 777/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:06:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:06:31.145150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee0718-62b7-43c2-aba8-d29dddcf980d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:31.145696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee0718-62b7-43c2-aba8-d29dddcf980d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:31.145696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee0718-62b7-43c2-aba8-d29dddcf980d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:31.147139 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Aug 19 21:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:06:31.153071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f133a68-4c8c-4741-8408-b7e5694ffe0d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:31.153523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f133a68-4c8c-4741-8408-b7e5694ffe0d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:31.153631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f133a68-4c8c-4741-8408-b7e5694ffe0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:31.172991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2f133a68-4c8c-4741-8408-b7e5694ffe0d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:06:31.174232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2f133a68-4c8c-4741-8408-b7e5694ffe0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:31.182135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Aug 19 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:06:31.739608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee0718-62b7-43c2-aba8-d29dddcf980d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:06:31.741469 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4dee0718-62b7-43c2-aba8-d29dddcf980d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:06:31.755487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee0718-62b7-43c2-aba8-d29dddcf980d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:06:31.759085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc76bee-c560-4575-a511-0e2d33da7958 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:06:31.759854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76bee-c560-4575-a511-0e2d33da7958 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:06:31.760082 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76bee-c560-4575-a511-0e2d33da7958 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:06:31.760367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc76bee-c560-4575-a511-0e2d33da7958 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:06:31.769599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc76bee-c560-4575-a511-0e2d33da7958 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:06:31.771067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc76bee-c560-4575-a511-0e2d33da7958 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:06:31.781149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 761/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:06:41.168308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=11 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:06:41.776890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:00.691376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc8b0-9827-4288-8583-27d63a0ece70 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:00.694000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc8b0-9827-4288-8583-27d63a0ece70 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:00.694000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fc8b0-9827-4288-8583-27d63a0ece70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:00.718507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-808fc8b0-9827-4288-8583-27d63a0ece70 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:00.720184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-808fc8b0-9827-4288-8583-27d63a0ece70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:00.731159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 779/1540] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:00.740990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b773044-c75d-4d12-8334-3fa1c0c14418 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:00.741660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b773044-c75d-4d12-8334-3fa1c0c14418 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:00.741660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b773044-c75d-4d12-8334-3fa1c0c14418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:00.780185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1b773044-c75d-4d12-8334-3fa1c0c14418 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:00.781787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1b773044-c75d-4d12-8334-3fa1c0c14418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:00.800023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 762/1541] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:00.801365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b20220-9d27-41fa-825d-312c1b0d0c4b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:00.801923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b20220-9d27-41fa-825d-312c1b0d0c4b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:00.802431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b20220-9d27-41fa-825d-312c1b0d0c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:00.822833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b773044-c75d-4d12-8334-3fa1c0c14418 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:00.837795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f4b20220-9d27-41fa-825d-312c1b0d0c4b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:00.837795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f4b20220-9d27-41fa-825d-312c1b0d0c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:00.840600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b773044-c75d-4d12-8334-3fa1c0c14418 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:00.858130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d72f4-daef-4c83-aee5-326633cf7cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:00.859091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 780/1542] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:00.859721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d72f4-daef-4c83-aee5-326633cf7cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:00.860236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d72f4-daef-4c83-aee5-326633cf7cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:00.860728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d72f4-daef-4c83-aee5-326633cf7cbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:00.861630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d72f4-daef-4c83-aee5-326633cf7cbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:00.862295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d72f4-daef-4c83-aee5-326633cf7cbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:00.863585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-597d72f4-daef-4c83-aee5-326633cf7cbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:08:00.863585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 19 21:08:00.863585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: Aug 19 21:08:00.864960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d72f4-daef-4c83-aee5-326633cf7cbd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:00.887174 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 763/1543] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:08:00.891960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c343b8-fd15-4b71-bc28-c64568c08baa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:00.891960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c343b8-fd15-4b71-bc28-c64568c08baa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:00.891960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c343b8-fd15-4b71-bc28-c64568c08baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:00.914767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d72f4-daef-4c83-aee5-326633cf7cbd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:00.927014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-79c343b8-fd15-4b71-bc28-c64568c08baa None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:00.928468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-79c343b8-fd15-4b71-bc28-c64568c08baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:00.932512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d72f4-daef-4c83-aee5-326633cf7cbd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:00.945297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 781/1544] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:00.947745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e419a46-b313-4657-88f7-f33af8c0a927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:00.948861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e419a46-b313-4657-88f7-f33af8c0a927 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:00.949377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e419a46-b313-4657-88f7-f33af8c0a927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:00.949979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e419a46-b313-4657-88f7-f33af8c0a927 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:00.950896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e419a46-b313-4657-88f7-f33af8c0a927 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:00.953095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c343b8-fd15-4b71-bc28-c64568c08baa None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:00.954131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e419a46-b313-4657-88f7-f33af8c0a927 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:00.968333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c343b8-fd15-4b71-bc28-c64568c08baa None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:00.984831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910a839-1fd9-4bd5-9afa-8b1af986c520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:00.985577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910a839-1fd9-4bd5-9afa-8b1af986c520 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:00.985737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910a839-1fd9-4bd5-9afa-8b1af986c520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:00.986033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910a839-1fd9-4bd5-9afa-8b1af986c520 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:00.986706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910a839-1fd9-4bd5-9afa-8b1af986c520 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:00.987011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910a839-1fd9-4bd5-9afa-8b1af986c520 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:00.988876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910a839-1fd9-4bd5-9afa-8b1af986c520 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:01.021404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 782/1545] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:08:01.041718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 764/1546] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:00 2022] POST /identity/v3/projects => generated 393 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:08:01.043964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910a839-1fd9-4bd5-9afa-8b1af986c520 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:01.055135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e419a46-b313-4657-88f7-f33af8c0a927 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:01.062172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910a839-1fd9-4bd5-9afa-8b1af986c520 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:01.160114 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e419a46-b313-4657-88f7-f33af8c0a927 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:01.174649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161e31b-40c4-43e0-8da6-05691b9fa902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:01.175515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161e31b-40c4-43e0-8da6-05691b9fa902 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.175984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161e31b-40c4-43e0-8da6-05691b9fa902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.176994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161e31b-40c4-43e0-8da6-05691b9fa902 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.177965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161e31b-40c4-43e0-8da6-05691b9fa902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:01.181805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-e161e31b-40c4-43e0-8da6-05691b9fa902 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:08:01.181805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 19 21:08:01.181805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: Aug 19 21:08:01.182279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161e31b-40c4-43e0-8da6-05691b9fa902 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:01.230584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 765/1547] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:08:01.232578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e4aa9-8664-490f-81ef-40263c9ee79a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.232907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e4aa9-8664-490f-81ef-40263c9ee79a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.233193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9e4aa9-8664-490f-81ef-40263c9ee79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.277632 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6a9e4aa9-8664-490f-81ef-40263c9ee79a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:01.277632 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6a9e4aa9-8664-490f-81ef-40263c9ee79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:01.295208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366df3b-2dc4-48b9-a4c5-8db223e77cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:01.296767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366df3b-2dc4-48b9-a4c5-8db223e77cad admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.297619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366df3b-2dc4-48b9-a4c5-8db223e77cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.303078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366df3b-2dc4-48b9-a4c5-8db223e77cad admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.305431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366df3b-2dc4-48b9-a4c5-8db223e77cad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:01.307755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-e366df3b-2dc4-48b9-a4c5-8db223e77cad admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:08:01.307755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 19 21:08:01.307755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: Aug 19 21:08:01.312118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 766/1548] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:01.314135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed503b-a742-4c3f-abf0-1bdd23c689b2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.314506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed503b-a742-4c3f-abf0-1bdd23c689b2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.314914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed503b-a742-4c3f-abf0-1bdd23c689b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.316048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366df3b-2dc4-48b9-a4c5-8db223e77cad admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:01.372939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5aed503b-a742-4c3f-abf0-1bdd23c689b2 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:01.375699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5aed503b-a742-4c3f-abf0-1bdd23c689b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:01.386933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 783/1549] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/projects => generated 379 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:01.389320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366df3b-2dc4-48b9-a4c5-8db223e77cad admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:01.398015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 767/1550] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:01.400090 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98cb55-9eac-4319-b275-a9abca511adf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.403621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98cb55-9eac-4319-b275-a9abca511adf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.405687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98cb55-9eac-4319-b275-a9abca511adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.451088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3e98cb55-9eac-4319-b275-a9abca511adf None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:01.452375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3e98cb55-9eac-4319-b275-a9abca511adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:01.569805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366df3b-2dc4-48b9-a4c5-8db223e77cad admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:01.582686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 768/1551] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:01.583568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98cb55-9eac-4319-b275-a9abca511adf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:01.584878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a70fa3-f0a5-4d4c-804d-0d399269956d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:01.586266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a70fa3-f0a5-4d4c-804d-0d399269956d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.586616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a70fa3-f0a5-4d4c-804d-0d399269956d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.586927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a70fa3-f0a5-4d4c-804d-0d399269956d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.587575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a70fa3-f0a5-4d4c-804d-0d399269956d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:01.589290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a70fa3-f0a5-4d4c-804d-0d399269956d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:01.607163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98cb55-9eac-4319-b275-a9abca511adf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:01.621413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 784/1552] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:01.623003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a610f3-599d-492a-af67-c0381934a555 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.623320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a610f3-599d-492a-af67-c0381934a555 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.623589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a610f3-599d-492a-af67-c0381934a555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.626055 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ab01c-1f0d-4f12-bab8-5f6e8aa170b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:01.627041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ab01c-1f0d-4f12-bab8-5f6e8aa170b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.627348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ab01c-1f0d-4f12-bab8-5f6e8aa170b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.627816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ab01c-1f0d-4f12-bab8-5f6e8aa170b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.628474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ab01c-1f0d-4f12-bab8-5f6e8aa170b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:01.633057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088ab01c-1f0d-4f12-bab8-5f6e8aa170b7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:01.676188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-20a610f3-599d-492a-af67-c0381934a555 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:01.678571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-20a610f3-599d-492a-af67-c0381934a555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:01.688217 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 769/1553] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:01.689853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a24cd3-3fff-425d-aa84-46e794c26861 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.690267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a24cd3-3fff-425d-aa84-46e794c26861 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.690663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a24cd3-3fff-425d-aa84-46e794c26861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.695538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 785/1554] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:01.697111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a610f3-599d-492a-af67-c0381934a555 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:01.723762 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-47a24cd3-3fff-425d-aa84-46e794c26861 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:01.727292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-47a24cd3-3fff-425d-aa84-46e794c26861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:01.730143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a610f3-599d-492a-af67-c0381934a555 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:01.736296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9866d8-d027-419b-97ae-45caed21d44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:01.737200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9866d8-d027-419b-97ae-45caed21d44b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.738168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9866d8-d027-419b-97ae-45caed21d44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.738704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9866d8-d027-419b-97ae-45caed21d44b admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.739615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9866d8-d027-419b-97ae-45caed21d44b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:01.740031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9866d8-d027-419b-97ae-45caed21d44b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:01.741277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 770/1555] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:01.742634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9866d8-d027-419b-97ae-45caed21d44b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:01.742794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a24cd3-3fff-425d-aa84-46e794c26861 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:01.750891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:01.762393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a24cd3-3fff-425d-aa84-46e794c26861 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:01.763342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 786/1556] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:08:01.765248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16bbc4-4b03-4977-bccf-657f318ef6e0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.765248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16bbc4-4b03-4977-bccf-657f318ef6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.765714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16bbc4-4b03-4977-bccf-657f318ef6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.773660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e8090-a182-42a5-bb20-b360d94dd32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:01.773660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e8090-a182-42a5-bb20-b360d94dd32e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.773660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e8090-a182-42a5-bb20-b360d94dd32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.773660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e8090-a182-42a5-bb20-b360d94dd32e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.774516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e8090-a182-42a5-bb20-b360d94dd32e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:01.774658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e8090-a182-42a5-bb20-b360d94dd32e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:01.790841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5d16bbc4-4b03-4977-bccf-657f318ef6e0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:01.792261 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 771/1557] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:01.793250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e8090-a182-42a5-bb20-b360d94dd32e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:01.795579 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5d16bbc4-4b03-4977-bccf-657f318ef6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:01.806874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 787/1558] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:01.807903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16bbc4-4b03-4977-bccf-657f318ef6e0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:01.812047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e8090-a182-42a5-bb20-b360d94dd32e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:01.818168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff63b5-e4ea-4895-963a-6009e3f20397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:01.818924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff63b5-e4ea-4895-963a-6009e3f20397 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.819213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff63b5-e4ea-4895-963a-6009e3f20397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.820394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff63b5-e4ea-4895-963a-6009e3f20397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.821316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff63b5-e4ea-4895-963a-6009e3f20397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:01.823489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff63b5-e4ea-4895-963a-6009e3f20397 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:01.832831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16bbc4-4b03-4977-bccf-657f318ef6e0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:01.840693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 772/1559] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:01.840693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bc942-7dc0-4513-ac16-a72154ab704d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.840693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bc942-7dc0-4513-ac16-a72154ab704d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.840693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bc942-7dc0-4513-ac16-a72154ab704d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.845723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33189ec8-861a-490e-a0ae-88915bc835b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:01.846955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33189ec8-861a-490e-a0ae-88915bc835b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.847763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33189ec8-861a-490e-a0ae-88915bc835b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.848172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33189ec8-861a-490e-a0ae-88915bc835b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.849473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33189ec8-861a-490e-a0ae-88915bc835b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:01.850482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33189ec8-861a-490e-a0ae-88915bc835b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:01.853936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33189ec8-861a-490e-a0ae-88915bc835b8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:01.865482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c76bc942-7dc0-4513-ac16-a72154ab704d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:01.867689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c76bc942-7dc0-4513-ac16-a72154ab704d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:01.877285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 773/1560] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:01.885690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 788/1561] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:08:01.885690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef2cc3-6747-4fca-a72d-6be65449e506 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.885690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef2cc3-6747-4fca-a72d-6be65449e506 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.885690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef2cc3-6747-4fca-a72d-6be65449e506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.887599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76bc942-7dc0-4513-ac16-a72154ab704d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:01.899288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76bc942-7dc0-4513-ac16-a72154ab704d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:01.904679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc5ebc-9e40-42f9-a3d5-543d8a08ac93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:01.905587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc5ebc-9e40-42f9-a3d5-543d8a08ac93 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.906116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc5ebc-9e40-42f9-a3d5-543d8a08ac93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.906580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc5ebc-9e40-42f9-a3d5-543d8a08ac93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.907668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc5ebc-9e40-42f9-a3d5-543d8a08ac93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:01.909813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc5ebc-9e40-42f9-a3d5-543d8a08ac93 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:01.921128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6bef2cc3-6747-4fca-a72d-6be65449e506 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:01.923277 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6bef2cc3-6747-4fca-a72d-6be65449e506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:01.939247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 789/1562] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:01.941003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef2cc3-6747-4fca-a72d-6be65449e506 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:01.946337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 774/1563] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:01.948451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364bf2e-20ba-4b09-8d23-c235d86b5da6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:01.948978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364bf2e-20ba-4b09-8d23-c235d86b5da6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:01.949395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364bf2e-20ba-4b09-8d23-c235d86b5da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:01.953120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:01.961008 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef2cc3-6747-4fca-a72d-6be65449e506 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:01.981869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e364bf2e-20ba-4b09-8d23-c235d86b5da6 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:01.984061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e364bf2e-20ba-4b09-8d23-c235d86b5da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:02.055498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291a26c-c576-40e6-a1dd-2166efe9054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.056766 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a26c-c576-40e6-a1dd-2166efe9054a admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.057272 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a26c-c576-40e6-a1dd-2166efe9054a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.057855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291a26c-c576-40e6-a1dd-2166efe9054a admin admin] PATH_INFO: `/v3/projects/1c4004ba973b4454865b889d6d1e049a/users/e8e345dbc2114b718c98c70844352c63/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.083729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 775/1564] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:02.085084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364bf2e-20ba-4b09-8d23-c235d86b5da6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.093559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291a26c-c576-40e6-a1dd-2166efe9054a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4004ba973b4454865b889d6d1e049a, user_id=e8e345dbc2114b718c98c70844352c63, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.095622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-4291a26c-c576-40e6-a1dd-2166efe9054a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:08:02.095622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 19 21:08:02.095622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: Aug 19 21:08:02.099777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291a26c-c576-40e6-a1dd-2166efe9054a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.102644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364bf2e-20ba-4b09-8d23-c235d86b5da6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.109575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47939a30-c64c-48b5-90ee-f643ed00407f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.110827 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47939a30-c64c-48b5-90ee-f643ed00407f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.111315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47939a30-c64c-48b5-90ee-f643ed00407f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.111854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47939a30-c64c-48b5-90ee-f643ed00407f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.112922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47939a30-c64c-48b5-90ee-f643ed00407f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.115104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47939a30-c64c-48b5-90ee-f643ed00407f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.130150 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 790/1565] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:01 2022] PUT /identity/v3/projects/1c4004ba973b4454865b889d6d1e049a/users/e8e345dbc2114b718c98c70844352c63/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:02.130983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 776/1566] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:02.131861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47939a30-c64c-48b5-90ee-f643ed00407f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.134159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e615d8-e84b-4312-9526-5b53175e1469 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.134482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e615d8-e84b-4312-9526-5b53175e1469 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.134867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e615d8-e84b-4312-9526-5b53175e1469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.148698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47939a30-c64c-48b5-90ee-f643ed00407f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.167042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d4e615d8-e84b-4312-9526-5b53175e1469 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:02.169209 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d4e615d8-e84b-4312-9526-5b53175e1469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:02.243637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee141ae-5011-4280-80c1-198e4a62c63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.244379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee141ae-5011-4280-80c1-198e4a62c63d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.244663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee141ae-5011-4280-80c1-198e4a62c63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.244936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee141ae-5011-4280-80c1-198e4a62c63d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.246114 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee141ae-5011-4280-80c1-198e4a62c63d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.247616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee141ae-5011-4280-80c1-198e4a62c63d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.263822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 791/1567] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:02.264614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e615d8-e84b-4312-9526-5b53175e1469 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.277737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e615d8-e84b-4312-9526-5b53175e1469 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.284472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f10c58-3964-4a06-9582-bc99b1b6a6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.285267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f10c58-3964-4a06-9582-bc99b1b6a6fb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.285624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f10c58-3964-4a06-9582-bc99b1b6a6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.286027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 777/1568] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/projects => generated 389 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:02.286236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f10c58-3964-4a06-9582-bc99b1b6a6fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.286595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee141ae-5011-4280-80c1-198e4a62c63d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.287308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f10c58-3964-4a06-9582-bc99b1b6a6fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:02.287889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f10c58-3964-4a06-9582-bc99b1b6a6fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.291501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f10c58-3964-4a06-9582-bc99b1b6a6fb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.314848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 792/1569] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:08:02.314848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f10c58-3964-4a06-9582-bc99b1b6a6fb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.321857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f10c58-3964-4a06-9582-bc99b1b6a6fb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.334914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:02.384489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee141ae-5011-4280-80c1-198e4a62c63d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.392068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b65c1-f564-4aab-8429-8cc1a202447d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.392755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b65c1-f564-4aab-8429-8cc1a202447d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.393066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b65c1-f564-4aab-8429-8cc1a202447d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.393354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37b65c1-f564-4aab-8429-8cc1a202447d admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.394074 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b65c1-f564-4aab-8429-8cc1a202447d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.396179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37b65c1-f564-4aab-8429-8cc1a202447d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.398281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8853d73-16ff-4386-8b07-2363737e61f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.399316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8853d73-16ff-4386-8b07-2363737e61f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.399817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8853d73-16ff-4386-8b07-2363737e61f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.400189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8853d73-16ff-4386-8b07-2363737e61f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.401013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8853d73-16ff-4386-8b07-2363737e61f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:02.401437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8853d73-16ff-4386-8b07-2363737e61f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.403704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8853d73-16ff-4386-8b07-2363737e61f9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.421283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 793/1570] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:08:02.423003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831994a-692a-49b9-a75d-a85d02f7488a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.428389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831994a-692a-49b9-a75d-a85d02f7488a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.428716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 778/1571] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:02.428906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831994a-692a-49b9-a75d-a85d02f7488a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.429609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b65c1-f564-4aab-8429-8cc1a202447d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.439477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37b65c1-f564-4aab-8429-8cc1a202447d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.444446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:02.446637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051f87a-1280-44ba-b1d9-35366b899c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.447578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051f87a-1280-44ba-b1d9-35366b899c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.447908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051f87a-1280-44ba-b1d9-35366b899c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.448266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051f87a-1280-44ba-b1d9-35366b899c57 admin admin] PATH_INFO: `/v3/projects/c0321d32b9014fc0ae63387d3a5032e8/users/593a0461ca714cff98ae43eb8833197c/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.452841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d831994a-692a-49b9-a75d-a85d02f7488a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:02.455269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d831994a-692a-49b9-a75d-a85d02f7488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:02.466300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 794/1572] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:02.467931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a450c3-4cec-4651-88cf-18ffec45bb79 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.468333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a450c3-4cec-4651-88cf-18ffec45bb79 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.468802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a450c3-4cec-4651-88cf-18ffec45bb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.473190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051f87a-1280-44ba-b1d9-35366b899c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0321d32b9014fc0ae63387d3a5032e8, user_id=593a0461ca714cff98ae43eb8833197c, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.476441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051f87a-1280-44ba-b1d9-35366b899c57 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.494592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-47a450c3-4cec-4651-88cf-18ffec45bb79 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:02.494969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 779/1573] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:02 2022] PUT /identity/v3/projects/c0321d32b9014fc0ae63387d3a5032e8/users/593a0461ca714cff98ae43eb8833197c/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:02.495913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051f87a-1280-44ba-b1d9-35366b899c57 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.496291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-47a450c3-4cec-4651-88cf-18ffec45bb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:02.586025 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 795/1574] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:02.587265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a450c3-4cec-4651-88cf-18ffec45bb79 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.591948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051f87a-1280-44ba-b1d9-35366b899c57 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.598055 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8358-37c3-483b-84d6-5d06ffd72aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.598678 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8358-37c3-483b-84d6-5d06ffd72aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.598955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8358-37c3-483b-84d6-5d06ffd72aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.599186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a450c3-4cec-4651-88cf-18ffec45bb79 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.599351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8358-37c3-483b-84d6-5d06ffd72aa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.599916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce8358-37c3-483b-84d6-5d06ffd72aa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:02.600223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce8358-37c3-483b-84d6-5d06ffd72aa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.603235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ce8358-37c3-483b-84d6-5d06ffd72aa7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.607810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9984063-286f-4df0-8868-e335d0564700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.608847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9984063-286f-4df0-8868-e335d0564700 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.609342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9984063-286f-4df0-8868-e335d0564700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.609855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9984063-286f-4df0-8868-e335d0564700 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.610955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9984063-286f-4df0-8868-e335d0564700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.614487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9984063-286f-4df0-8868-e335d0564700 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.627574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 780/1575] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:08:02.629079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8358-37c3-483b-84d6-5d06ffd72aa7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.648677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8358-37c3-483b-84d6-5d06ffd72aa7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.657348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620a4fc-bf02-4476-8f34-15bcdd205dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.657736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 796/1576] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/users => generated 295 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:02.658467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620a4fc-bf02-4476-8f34-15bcdd205dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.659207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9984063-286f-4df0-8868-e335d0564700 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.659395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620a4fc-bf02-4476-8f34-15bcdd205dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.659699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620a4fc-bf02-4476-8f34-15bcdd205dc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.660786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620a4fc-bf02-4476-8f34-15bcdd205dc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.666127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620a4fc-bf02-4476-8f34-15bcdd205dc4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.678868 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9984063-286f-4df0-8868-e335d0564700 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.690767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a98d123-feda-4448-9551-41b9e370741d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.691909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98d123-feda-4448-9551-41b9e370741d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.692409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98d123-feda-4448-9551-41b9e370741d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.693020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98d123-feda-4448-9551-41b9e370741d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.694418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98d123-feda-4448-9551-41b9e370741d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.697049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a98d123-feda-4448-9551-41b9e370741d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.715188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 781/1577] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:02.715188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620a4fc-bf02-4476-8f34-15bcdd205dc4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.716335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 797/1578] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:02.718264 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2446c0a-bde1-4dc1-89e9-64f516480b98 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.718763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2446c0a-bde1-4dc1-89e9-64f516480b98 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.719154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2446c0a-bde1-4dc1-89e9-64f516480b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.747327 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c2446c0a-bde1-4dc1-89e9-64f516480b98 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:02.749029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c2446c0a-bde1-4dc1-89e9-64f516480b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:02.824195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620a4fc-bf02-4476-8f34-15bcdd205dc4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:02.832450 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ebea2-f3fd-4957-97d6-12669b154ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:02.833522 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ebea2-f3fd-4957-97d6-12669b154ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.834057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ebea2-f3fd-4957-97d6-12669b154ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.834538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ebea2-f3fd-4957-97d6-12669b154ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.835778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ebea2-f3fd-4957-97d6-12669b154ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:02.839422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ebea2-f3fd-4957-97d6-12669b154ee3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:02.847311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 798/1579] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:02.849517 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7e8c6-c00e-4a8c-b7cb-88f7e7c8357e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:02.850051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7e8c6-c00e-4a8c-b7cb-88f7e7c8357e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:02.850547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e7e8c6-c00e-4a8c-b7cb-88f7e7c8357e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:02.858917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:02.877148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-87e7e8c6-c00e-4a8c-b7cb-88f7e7c8357e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:02.878578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-87e7e8c6-c00e-4a8c-b7cb-88f7e7c8357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:02.888758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 799/1580] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:02.889636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7e8c6-c00e-4a8c-b7cb-88f7e7c8357e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.899325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 782/1581] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/projects => generated 403 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:02.899325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ebea2-f3fd-4957-97d6-12669b154ee3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:02.962281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e7e8c6-c00e-4a8c-b7cb-88f7e7c8357e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.000479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:03.005035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ebea2-f3fd-4957-97d6-12669b154ee3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.010962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9dece-390a-4b23-b548-03ae5b20248e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.011652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9dece-390a-4b23-b548-03ae5b20248e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.011930 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9dece-390a-4b23-b548-03ae5b20248e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.012205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9dece-390a-4b23-b548-03ae5b20248e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.012869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9dece-390a-4b23-b548-03ae5b20248e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.014960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9dece-390a-4b23-b548-03ae5b20248e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.054399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a0523-6571-4b68-9c30-db723394f996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.055389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a0523-6571-4b68-9c30-db723394f996 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.055844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a0523-6571-4b68-9c30-db723394f996 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.056311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a0523-6571-4b68-9c30-db723394f996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.056511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 783/1582] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:02 2022] POST /identity/v3/projects => generated 385 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:03.057845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9dece-390a-4b23-b548-03ae5b20248e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.074682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a0523-6571-4b68-9c30-db723394f996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.076608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56a0523-6571-4b68-9c30-db723394f996 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.091219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 800/1583] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:03.092687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a0523-6571-4b68-9c30-db723394f996 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.112134 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56a0523-6571-4b68-9c30-db723394f996 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.180235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9dece-390a-4b23-b548-03ae5b20248e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.191198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206acb0b-3d51-415d-a660-0ed9c3b23e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.192341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206acb0b-3d51-415d-a660-0ed9c3b23e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.193134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206acb0b-3d51-415d-a660-0ed9c3b23e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.193328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206acb0b-3d51-415d-a660-0ed9c3b23e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.196356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206acb0b-3d51-415d-a660-0ed9c3b23e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.199865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206acb0b-3d51-415d-a660-0ed9c3b23e2c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.226616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 784/1584] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:03.226616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206acb0b-3d51-415d-a660-0ed9c3b23e2c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.230174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5021c-a331-4353-bbb6-701bdfa738ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.231049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5021c-a331-4353-bbb6-701bdfa738ac admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.231435 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5021c-a331-4353-bbb6-701bdfa738ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.231814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5021c-a331-4353-bbb6-701bdfa738ac admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.232520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5021c-a331-4353-bbb6-701bdfa738ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.235036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5021c-a331-4353-bbb6-701bdfa738ac admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.249403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206acb0b-3d51-415d-a660-0ed9c3b23e2c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.261183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b187e-a545-49d4-9815-227bbe0dbee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.262290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b187e-a545-49d4-9815-227bbe0dbee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.263060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b187e-a545-49d4-9815-227bbe0dbee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.263248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b187e-a545-49d4-9815-227bbe0dbee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.265057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b187e-a545-49d4-9815-227bbe0dbee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.266759 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 801/1585] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:03 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:03.268255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5021c-a331-4353-bbb6-701bdfa738ac admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.269565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b187e-a545-49d4-9815-227bbe0dbee1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.283837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5021c-a331-4353-bbb6-701bdfa738ac admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.295474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 785/1586] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:03.295474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b187e-a545-49d4-9815-227bbe0dbee1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.298510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fb1d4-7df2-4a0d-86c5-ef02de337318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.299511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fb1d4-7df2-4a0d-86c5-ef02de337318 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.299826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fb1d4-7df2-4a0d-86c5-ef02de337318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.300115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fb1d4-7df2-4a0d-86c5-ef02de337318 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.300911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fb1d4-7df2-4a0d-86c5-ef02de337318 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:03.301329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fb1d4-7df2-4a0d-86c5-ef02de337318 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.303787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367fb1d4-7df2-4a0d-86c5-ef02de337318 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.319292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b187e-a545-49d4-9815-227bbe0dbee1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.325193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 802/1587] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:08:03.326603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fb1d4-7df2-4a0d-86c5-ef02de337318 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.330677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d28fc-452f-41dd-a458-0ad3992376eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.331992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d28fc-452f-41dd-a458-0ad3992376eb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.332609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d28fc-452f-41dd-a458-0ad3992376eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.333216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d28fc-452f-41dd-a458-0ad3992376eb admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.334523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d28fc-452f-41dd-a458-0ad3992376eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.339223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d28fc-452f-41dd-a458-0ad3992376eb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.377841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 786/1588] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:03 2022] POST /identity/v3/users => generated 302 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:03.377841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d28fc-452f-41dd-a458-0ad3992376eb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.393894 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d28fc-452f-41dd-a458-0ad3992376eb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.403057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94257602-c560-48b2-855e-e31226ac1f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.406237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94257602-c560-48b2-855e-e31226ac1f89 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.406237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94257602-c560-48b2-855e-e31226ac1f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.406237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94257602-c560-48b2-855e-e31226ac1f89 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.406578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94257602-c560-48b2-855e-e31226ac1f89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.410175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94257602-c560-48b2-855e-e31226ac1f89 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.442800 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fb1d4-7df2-4a0d-86c5-ef02de337318 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.453829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 787/1589] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:03 2022] POST /identity/v3/users => generated 293 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:03.454088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cbaa7-afe4-44b1-9211-b2b4901ee246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.454197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94257602-c560-48b2-855e-e31226ac1f89 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.454276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cbaa7-afe4-44b1-9211-b2b4901ee246 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.454276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cbaa7-afe4-44b1-9211-b2b4901ee246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.454430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cbaa7-afe4-44b1-9211-b2b4901ee246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.474940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cbaa7-afe4-44b1-9211-b2b4901ee246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.475480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94257602-c560-48b2-855e-e31226ac1f89 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.477002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cbaa7-afe4-44b1-9211-b2b4901ee246 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.484646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd503b-cf82-48e9-9dae-f1eae4e3dc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.485507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd503b-cf82-48e9-9dae-f1eae4e3dc39 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.485686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd503b-cf82-48e9-9dae-f1eae4e3dc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.485925 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd503b-cf82-48e9-9dae-f1eae4e3dc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.486506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd503b-cf82-48e9-9dae-f1eae4e3dc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.488169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd503b-cf82-48e9-9dae-f1eae4e3dc39 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.498537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 803/1590] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:03.500642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cbaa7-afe4-44b1-9211-b2b4901ee246 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.509789 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 788/1591] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:03.512115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd503b-cf82-48e9-9dae-f1eae4e3dc39 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.522692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cbaa7-afe4-44b1-9211-b2b4901ee246 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.535199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8d548-2c5a-4f0b-9a40-4f8e757001f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.542490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8d548-2c5a-4f0b-9a40-4f8e757001f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.543216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8d548-2c5a-4f0b-9a40-4f8e757001f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.543769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8d548-2c5a-4f0b-9a40-4f8e757001f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.545170 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8d548-2c5a-4f0b-9a40-4f8e757001f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.546420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd503b-cf82-48e9-9dae-f1eae4e3dc39 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.548974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8d548-2c5a-4f0b-9a40-4f8e757001f8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.557990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f4182-e1e5-4b20-86e2-f2c3a8c4d6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.559151 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f4182-e1e5-4b20-86e2-f2c3a8c4d6f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.559657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f4182-e1e5-4b20-86e2-f2c3a8c4d6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.560174 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f4182-e1e5-4b20-86e2-f2c3a8c4d6f4 admin admin] PATH_INFO: `/v3/projects/6f7cafd832804f8bab8322ee27c1803d/users/629d7d6f348d4805bd10850d151daefb/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.585675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 804/1592] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:03.585675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8d548-2c5a-4f0b-9a40-4f8e757001f8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.586802 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:03.596857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f4182-e1e5-4b20-86e2-f2c3a8c4d6f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f7cafd832804f8bab8322ee27c1803d, user_id=629d7d6f348d4805bd10850d151daefb, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.600260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f4182-e1e5-4b20-86e2-f2c3a8c4d6f4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.606010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8d548-2c5a-4f0b-9a40-4f8e757001f8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.617354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5e59b-f988-4dd3-b046-abb3ab2d3017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.618471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5e59b-f988-4dd3-b046-abb3ab2d3017 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.619007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5e59b-f988-4dd3-b046-abb3ab2d3017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.619669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5e59b-f988-4dd3-b046-abb3ab2d3017 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.620840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5e59b-f988-4dd3-b046-abb3ab2d3017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.625428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5e59b-f988-4dd3-b046-abb3ab2d3017 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.628775 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 789/1593] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:03 2022] PUT /identity/v3/projects/6f7cafd832804f8bab8322ee27c1803d/users/629d7d6f348d4805bd10850d151daefb/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:03.640885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f4182-e1e5-4b20-86e2-f2c3a8c4d6f4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.687875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 805/1594] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:03 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:03.688869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5e59b-f988-4dd3-b046-abb3ab2d3017 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.710133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:03.751299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f4182-e1e5-4b20-86e2-f2c3a8c4d6f4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.803063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5e59b-f988-4dd3-b046-abb3ab2d3017 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.810019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48b4fa-b9c0-4d1d-a452-8d9ea33a8c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.810603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48b4fa-b9c0-4d1d-a452-8d9ea33a8c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.810878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48b4fa-b9c0-4d1d-a452-8d9ea33a8c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.811198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48b4fa-b9c0-4d1d-a452-8d9ea33a8c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.812112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48b4fa-b9c0-4d1d-a452-8d9ea33a8c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.814217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc48b4fa-b9c0-4d1d-a452-8d9ea33a8c19 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.828030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 806/1595] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:03.833924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48b4fa-b9c0-4d1d-a452-8d9ea33a8c19 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.848702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc48b4fa-b9c0-4d1d-a452-8d9ea33a8c19 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.849882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdc9b5-82a8-4120-b589-e4fe7410690e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.851001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdc9b5-82a8-4120-b589-e4fe7410690e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.851674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdc9b5-82a8-4120-b589-e4fe7410690e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.852267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdc9b5-82a8-4120-b589-e4fe7410690e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.853514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdc9b5-82a8-4120-b589-e4fe7410690e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.855991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bee4ca-12f8-477f-aced-113c8ac2c827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.856533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdc9b5-82a8-4120-b589-e4fe7410690e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.856823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee4ca-12f8-477f-aced-113c8ac2c827 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.857109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee4ca-12f8-477f-aced-113c8ac2c827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.857391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bee4ca-12f8-477f-aced-113c8ac2c827 admin admin] PATH_INFO: `/v3/projects/a3902dce55324b7ab42df9de97bc11b4/users/0809d96091c2476c83ec0c7c6c5a1509/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.886710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 790/1596] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:03.886710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdc9b5-82a8-4120-b589-e4fe7410690e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:03.908481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdc9b5-82a8-4120-b589-e4fe7410690e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:03.912625 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bee4ca-12f8-477f-aced-113c8ac2c827 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3902dce55324b7ab42df9de97bc11b4, user_id=0809d96091c2476c83ec0c7c6c5a1509, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.918867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2b7c3-957a-4b1d-9814-cf5cc1dfad24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:03.920256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2b7c3-957a-4b1d-9814-cf5cc1dfad24 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.921163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2b7c3-957a-4b1d-9814-cf5cc1dfad24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.921606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bee4ca-12f8-477f-aced-113c8ac2c827 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.921855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2b7c3-957a-4b1d-9814-cf5cc1dfad24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.925131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2b7c3-957a-4b1d-9814-cf5cc1dfad24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:03.926600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2b7c3-957a-4b1d-9814-cf5cc1dfad24 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:03.952682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 791/1597] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:03.959578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e9b6d-c5e4-4ab4-8946-96e653235da5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:03.959578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e9b6d-c5e4-4ab4-8946-96e653235da5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:03.960603 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790e9b6d-c5e4-4ab4-8946-96e653235da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:03.975697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 807/1598] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:03 2022] PUT /identity/v3/projects/a3902dce55324b7ab42df9de97bc11b4/users/0809d96091c2476c83ec0c7c6c5a1509/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:03.981366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bee4ca-12f8-477f-aced-113c8ac2c827 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:04.014686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-790e9b6d-c5e4-4ab4-8946-96e653235da5 None None] MFA Rules not processed for user `629d7d6f348d4805bd10850d151daefb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:04.016277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-790e9b6d-c5e4-4ab4-8946-96e653235da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:04.098857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bee4ca-12f8-477f-aced-113c8ac2c827 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:04.106286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597e7e5-ac68-463d-a4ab-884897370e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:04.107130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597e7e5-ac68-463d-a4ab-884897370e02 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.107581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597e7e5-ac68-463d-a4ab-884897370e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.108191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597e7e5-ac68-463d-a4ab-884897370e02 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.109080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597e7e5-ac68-463d-a4ab-884897370e02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:04.111401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597e7e5-ac68-463d-a4ab-884897370e02 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:04.158325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 808/1599] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:03 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:04.163703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597e7e5-ac68-463d-a4ab-884897370e02 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:04.178336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597e7e5-ac68-463d-a4ab-884897370e02 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:04.186381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fdefe-d7dd-4a4b-9169-bde8c1caff86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:04.187418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fdefe-d7dd-4a4b-9169-bde8c1caff86 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.187993 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fdefe-d7dd-4a4b-9169-bde8c1caff86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.188456 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91fdefe-d7dd-4a4b-9169-bde8c1caff86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.189408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fdefe-d7dd-4a4b-9169-bde8c1caff86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:04.191827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91fdefe-d7dd-4a4b-9169-bde8c1caff86 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:04.216402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 809/1600] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:04.218389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fdefe-d7dd-4a4b-9169-bde8c1caff86 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:04.235689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 792/1601] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:04.237825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e9b6d-c5e4-4ab4-8946-96e653235da5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:04.246773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91fdefe-d7dd-4a4b-9169-bde8c1caff86 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:04.255152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f323e1e-0c4a-4130-842e-9dd76ee79fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:04.256789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f323e1e-0c4a-4130-842e-9dd76ee79fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.258166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f323e1e-0c4a-4130-842e-9dd76ee79fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.258980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f323e1e-0c4a-4130-842e-9dd76ee79fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.260112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f323e1e-0c4a-4130-842e-9dd76ee79fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:04.262047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f323e1e-0c4a-4130-842e-9dd76ee79fc1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:04.265725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790e9b6d-c5e4-4ab4-8946-96e653235da5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:04.278169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 810/1602] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:04.279685 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc7cd2-9772-4389-a843-9b827cebcebc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.281281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc7cd2-9772-4389-a843-9b827cebcebc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.281281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc7cd2-9772-4389-a843-9b827cebcebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.286843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b658322-ac3d-4dd0-93d1-e3849531f63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:04.289063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b658322-ac3d-4dd0-93d1-e3849531f63e admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.291198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b658322-ac3d-4dd0-93d1-e3849531f63e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.293279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b658322-ac3d-4dd0-93d1-e3849531f63e admin admin] PATH_INFO: `/v3/projects/358e50f9b9df43b7aa0a4ba1353e4a25/users/7f46019e78cc4e89bf877a4f5a7a8fe9/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.303880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fedc7cd2-9772-4389-a843-9b827cebcebc None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:04.306314 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fedc7cd2-9772-4389-a843-9b827cebcebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:04.316933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 811/1603] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:04.318182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc7cd2-9772-4389-a843-9b827cebcebc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:04.328445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:04.334735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc7cd2-9772-4389-a843-9b827cebcebc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:04.344254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cc73a-c0e7-4779-b2fb-2ba99f00ada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:04.345019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cc73a-c0e7-4779-b2fb-2ba99f00ada7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.345413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cc73a-c0e7-4779-b2fb-2ba99f00ada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.345782 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cc73a-c0e7-4779-b2fb-2ba99f00ada7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.345994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b658322-ac3d-4dd0-93d1-e3849531f63e admin admin] RBAC: Authorizing `identity:create_grant(project_id=358e50f9b9df43b7aa0a4ba1353e4a25, user_id=7f46019e78cc4e89bf877a4f5a7a8fe9, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:04.346658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cc73a-c0e7-4779-b2fb-2ba99f00ada7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:04.348559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780cc73a-c0e7-4779-b2fb-2ba99f00ada7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:04.357642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b658322-ac3d-4dd0-93d1-e3849531f63e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:04.367666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 812/1604] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:04.369586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cc73a-c0e7-4779-b2fb-2ba99f00ada7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:04.381169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780cc73a-c0e7-4779-b2fb-2ba99f00ada7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:04.390945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 793/1605] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:04 2022] PUT /identity/v3/projects/358e50f9b9df43b7aa0a4ba1353e4a25/users/7f46019e78cc4e89bf877a4f5a7a8fe9/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:04.394344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b658322-ac3d-4dd0-93d1-e3849531f63e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:04.436609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:04.494761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ba9cc-42c0-455e-b3fc-939deb6162e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:04.495720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ba9cc-42c0-455e-b3fc-939deb6162e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.496258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ba9cc-42c0-455e-b3fc-939deb6162e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.496878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ba9cc-42c0-455e-b3fc-939deb6162e4 admin admin] PATH_INFO: `/v3/projects/c654ccea4cb549a5848505a75432b760/users/65aaea0b67d94ad2a80a5acb7b4a3bec/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.562922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ba9cc-42c0-455e-b3fc-939deb6162e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c654ccea4cb549a5848505a75432b760, user_id=65aaea0b67d94ad2a80a5acb7b4a3bec, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:04.569049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ba9cc-42c0-455e-b3fc-939deb6162e4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:04.594353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b658322-ac3d-4dd0-93d1-e3849531f63e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:04.600133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 813/1606] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:04 2022] PUT /identity/v3/projects/c654ccea4cb549a5848505a75432b760/users/65aaea0b67d94ad2a80a5acb7b4a3bec/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:04.604609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d45498-6c39-4ec8-bc33-e5b25efc800f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.606433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d45498-6c39-4ec8-bc33-e5b25efc800f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.607604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d45498-6c39-4ec8-bc33-e5b25efc800f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.611042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 814/1607] 10.222.0.16 () {54 vars in 962 bytes} [Fri Aug 19 21:08:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:08:04.613001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d45498-6c39-4ec8-bc33-e5b25efc800f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:04.808908 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d45498-6c39-4ec8-bc33-e5b25efc800f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:04.828320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc63af-5f65-441b-a757-ff44f359424f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:04.829025 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc63af-5f65-441b-a757-ff44f359424f service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.829315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc63af-5f65-441b-a757-ff44f359424f service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.829851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc63af-5f65-441b-a757-ff44f359424f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.832729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904ab3c-ae0d-43e1-b3a3-7c3d1b4c4bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:04.837725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904ab3c-ae0d-43e1-b3a3-7c3d1b4c4bad admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.838432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904ab3c-ae0d-43e1-b3a3-7c3d1b4c4bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.839280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904ab3c-ae0d-43e1-b3a3-7c3d1b4c4bad admin admin] PATH_INFO: `/v3/projects/903bec1eca3c4c69b74e6f2961c31d9a/users/4c338e58dc874c61b78364907397a85b/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.863780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc63af-5f65-441b-a757-ff44f359424f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:04.867420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc63af-5f65-441b-a757-ff44f359424f service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:04.897661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 815/1608] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:04.899014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc63af-5f65-441b-a757-ff44f359424f service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:04.916389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904ab3c-ae0d-43e1-b3a3-7c3d1b4c4bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=903bec1eca3c4c69b74e6f2961c31d9a, user_id=4c338e58dc874c61b78364907397a85b, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:04.919654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc63af-5f65-441b-a757-ff44f359424f service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:04.924638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904ab3c-ae0d-43e1-b3a3-7c3d1b4c4bad admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:04.933669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830f461b-2855-4306-9d97-7327a457e6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:04.934546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f461b-2855-4306-9d97-7327a457e6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.935040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f461b-2855-4306-9d97-7327a457e6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.935528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830f461b-2855-4306-9d97-7327a457e6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.941192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f461b-2855-4306-9d97-7327a457e6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:04.944537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830f461b-2855-4306-9d97-7327a457e6b7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:04.966939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 794/1609] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:04 2022] PUT /identity/v3/projects/903bec1eca3c4c69b74e6f2961c31d9a/users/4c338e58dc874c61b78364907397a85b/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:04.974901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8df95-6eec-4b00-85a3-be9e6a50e187 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.976552 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8df95-6eec-4b00-85a3-be9e6a50e187 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.977950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8df95-6eec-4b00-85a3-be9e6a50e187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:04.983801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 816/1610] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:04.985306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a17bd-1fb7-474e-8aef-27d2a88064a6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:04.985596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a17bd-1fb7-474e-8aef-27d2a88064a6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:04.986049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a17bd-1fb7-474e-8aef-27d2a88064a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:05.047598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-08a8df95-6eec-4b00-85a3-be9e6a50e187 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:05.050745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-08a8df95-6eec-4b00-85a3-be9e6a50e187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:05.125582 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cb8a17bd-1fb7-474e-8aef-27d2a88064a6 None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:05.130914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cb8a17bd-1fb7-474e-8aef-27d2a88064a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:05.271791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 795/1611] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:05.275598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbe476-1ce1-4d7e-8a70-d536851d532d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:05.276319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbe476-1ce1-4d7e-8a70-d536851d532d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:05.276971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cbe476-1ce1-4d7e-8a70-d536851d532d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:05.355972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-28cbe476-1ce1-4d7e-8a70-d536851d532d None None] MFA Rules not processed for user `65aaea0b67d94ad2a80a5acb7b4a3bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:05.359621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-28cbe476-1ce1-4d7e-8a70-d536851d532d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:05.371448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:05.388892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 817/1612] 10.222.0.16 () {58 vars in 1059 bytes} [Fri Aug 19 21:08:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:05.391641 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183727ca-bfa8-45e6-97a6-1a4381cf35f9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:05.392129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183727ca-bfa8-45e6-97a6-1a4381cf35f9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:05.392705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183727ca-bfa8-45e6-97a6-1a4381cf35f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:05.470180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-183727ca-bfa8-45e6-97a6-1a4381cf35f9 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:05.471771 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-183727ca-bfa8-45e6-97a6-1a4381cf35f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:05.497312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 818/1613] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:05.499316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183727ca-bfa8-45e6-97a6-1a4381cf35f9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:05.551087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183727ca-bfa8-45e6-97a6-1a4381cf35f9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:05.570288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 796/1614] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:05.572127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbe476-1ce1-4d7e-8a70-d536851d532d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:05.591982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79fa41-609a-4c8e-a084-a11909afa268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:05.596729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fa41-609a-4c8e-a084-a11909afa268 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:05.599883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fa41-609a-4c8e-a084-a11909afa268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:05.601749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79fa41-609a-4c8e-a084-a11909afa268 admin admin] PATH_INFO: `/v3/projects/def1895e8f3b40f1b608240c60a0066a/users/8810b3e211d448cea9a506d9183dc13d/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:05.677378 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79fa41-609a-4c8e-a084-a11909afa268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=def1895e8f3b40f1b608240c60a0066a, user_id=8810b3e211d448cea9a506d9183dc13d, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:05.680726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79fa41-609a-4c8e-a084-a11909afa268 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:05.692605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:05.711639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 819/1615] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:05 2022] PUT /identity/v3/projects/def1895e8f3b40f1b608240c60a0066a/users/8810b3e211d448cea9a506d9183dc13d/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:05.719901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79fa41-609a-4c8e-a084-a11909afa268 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:05.755440 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cbe476-1ce1-4d7e-8a70-d536851d532d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:05.854095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79fa41-609a-4c8e-a084-a11909afa268 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:05.875719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4db1f0-45fd-45e5-8be0-7d2e2a391c99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:05.876492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4db1f0-45fd-45e5-8be0-7d2e2a391c99 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:05.877044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4db1f0-45fd-45e5-8be0-7d2e2a391c99 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:05.877386 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4db1f0-45fd-45e5-8be0-7d2e2a391c99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:05.908106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b04e86-0d30-494e-af7c-4e7b9cb61baf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:05.909517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b04e86-0d30-494e-af7c-4e7b9cb61baf service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:05.910287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b04e86-0d30-494e-af7c-4e7b9cb61baf service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:05.910796 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b04e86-0d30-494e-af7c-4e7b9cb61baf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:06.032892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4db1f0-45fd-45e5-8be0-7d2e2a391c99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:06.034596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4db1f0-45fd-45e5-8be0-7d2e2a391c99 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:06.066432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 820/1616] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:06.069343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4db1f0-45fd-45e5-8be0-7d2e2a391c99 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:06.105673 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b04e86-0d30-494e-af7c-4e7b9cb61baf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:06.111110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b04e86-0d30-494e-af7c-4e7b9cb61baf service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:06.117116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4db1f0-45fd-45e5-8be0-7d2e2a391c99 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:06.140845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 797/1617] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:06.143520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60543a1c-801f-433d-957c-2aefba9a3780 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:06.143926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce70bab-72ee-447a-ba6c-72afdb681b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:06.144143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60543a1c-801f-433d-957c-2aefba9a3780 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:06.144757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60543a1c-801f-433d-957c-2aefba9a3780 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:06.144940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce70bab-72ee-447a-ba6c-72afdb681b9a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:06.145205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce70bab-72ee-447a-ba6c-72afdb681b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:06.145481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce70bab-72ee-447a-ba6c-72afdb681b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:06.149366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 798/1618] 10.222.0.16 () {54 vars in 962 bytes} [Fri Aug 19 21:08:06 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:08:06.153205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69b0f9-18bd-4d19-9eea-c88fee7362b0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:06.153873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69b0f9-18bd-4d19-9eea-c88fee7362b0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:06.154330 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69b0f9-18bd-4d19-9eea-c88fee7362b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:06.191066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce70bab-72ee-447a-ba6c-72afdb681b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:06.192810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce70bab-72ee-447a-ba6c-72afdb681b9a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:06.207497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ed69b0f9-18bd-4d19-9eea-c88fee7362b0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:06.209245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ed69b0f9-18bd-4d19-9eea-c88fee7362b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:06.225238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 799/1619] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:06.228397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1e19c-3888-4b06-a1e5-d12c5eb50d8f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:06.228898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1e19c-3888-4b06-a1e5-d12c5eb50d8f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:06.229036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 821/1620] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:06.229511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1e19c-3888-4b06-a1e5-d12c5eb50d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:06.258712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:06.270525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e2d1e19c-3888-4b06-a1e5-d12c5eb50d8f None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:06.279206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e2d1e19c-3888-4b06-a1e5-d12c5eb50d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:06.308249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 800/1621] 10.222.0.16 () {58 vars in 1059 bytes} [Fri Aug 19 21:08:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:06.327605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce70bab-72ee-447a-ba6c-72afdb681b9a service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:06.378570 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce70bab-72ee-447a-ba6c-72afdb681b9a service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:06.393344 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a20b6-8919-4e0e-9c49-4b84c1015d2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:06.394552 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a20b6-8919-4e0e-9c49-4b84c1015d2f service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:06.395093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a20b6-8919-4e0e-9c49-4b84c1015d2f service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:06.395706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a20b6-8919-4e0e-9c49-4b84c1015d2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:06.573617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a03f8c-3892-4b73-9b10-2c7ed48943db None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:06.573617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a03f8c-3892-4b73-9b10-2c7ed48943db None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:06.573617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a03f8c-3892-4b73-9b10-2c7ed48943db None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:06.573617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 801/1622] 10.222.0.16 () {54 vars in 912 bytes} [Fri Aug 19 21:08:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:08:06.580255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0cf60-7fda-4333-891a-b4fee86d4d73 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:06.580933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a20b6-8919-4e0e-9c49-4b84c1015d2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:06.581130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0cf60-7fda-4333-891a-b4fee86d4d73 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:06.581517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0cf60-7fda-4333-891a-b4fee86d4d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:06.584514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a20b6-8919-4e0e-9c49-4b84c1015d2f service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:06.610359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 822/1623] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:06.625006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a20b6-8919-4e0e-9c49-4b84c1015d2f service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:06.651142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-68d0cf60-7fda-4333-891a-b4fee86d4d73 None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:06.652705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-68d0cf60-7fda-4333-891a-b4fee86d4d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:06.675833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a20b6-8919-4e0e-9c49-4b84c1015d2f service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:06.687599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 802/1624] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:08:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:06.690031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0cf60-7fda-4333-891a-b4fee86d4d73 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:06.693049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb32075-abcf-4b3e-a44e-213b229d634a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:06.694455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb32075-abcf-4b3e-a44e-213b229d634a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:06.698113 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb32075-abcf-4b3e-a44e-213b229d634a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:06.699208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb32075-abcf-4b3e-a44e-213b229d634a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:06.734573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb32075-abcf-4b3e-a44e-213b229d634a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:06.738753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb32075-abcf-4b3e-a44e-213b229d634a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:06.780596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 823/1625] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:06.822847 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0cf60-7fda-4333-891a-b4fee86d4d73 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:06.840142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f3fc2-01dd-4131-a98a-ea736238eeae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:06.841227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f3fc2-01dd-4131-a98a-ea736238eeae service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:06.841706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f3fc2-01dd-4131-a98a-ea736238eeae service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:06.842305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f3fc2-01dd-4131-a98a-ea736238eeae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:06.909829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f3fc2-01dd-4131-a98a-ea736238eeae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:06.912755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f3fc2-01dd-4131-a98a-ea736238eeae service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:06.976845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 803/1626] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:07.501292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b6139-d974-44fa-9a10-cbc346735ee5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:07.503041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b6139-d974-44fa-9a10-cbc346735ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:07.505262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b6139-d974-44fa-9a10-cbc346735ee5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:07.509537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 824/1627] 10.222.0.16 () {54 vars in 912 bytes} [Fri Aug 19 21:08:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:08:07.519176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f41fb-99d5-4305-8f78-fdc3d03fe59e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:07.520028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f41fb-99d5-4305-8f78-fdc3d03fe59e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:07.520028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f41fb-99d5-4305-8f78-fdc3d03fe59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:07.583004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-008f41fb-99d5-4305-8f78-fdc3d03fe59e None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:07.588492 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-008f41fb-99d5-4305-8f78-fdc3d03fe59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:07.643534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 804/1628] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:08:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:07.671141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b6139-d974-44fa-9a10-cbc346735ee5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:07.697085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:07.714574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b6139-d974-44fa-9a10-cbc346735ee5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:07.730463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661eb79-1730-4e51-a4e6-a38f5691e03a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:07.734372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661eb79-1730-4e51-a4e6-a38f5691e03a service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:07.734745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661eb79-1730-4e51-a4e6-a38f5691e03a service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:07.736536 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661eb79-1730-4e51-a4e6-a38f5691e03a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:07.791129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661eb79-1730-4e51-a4e6-a38f5691e03a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:07.794209 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8661eb79-1730-4e51-a4e6-a38f5691e03a service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:07.854426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 825/1629] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:13.916734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95eac7-b831-4e92-a9ef-51d81862172f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:13.917185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95eac7-b831-4e92-a9ef-51d81862172f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:13.917185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a95eac7-b831-4e92-a9ef-51d81862172f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:14.047747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8a95eac7-b831-4e92-a9ef-51d81862172f None None] MFA Rules not processed for user `4c338e58dc874c61b78364907397a85b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:14.065282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8a95eac7-b831-4e92-a9ef-51d81862172f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:14.305430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 805/1630] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:13 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:14.308051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661eb79-1730-4e51-a4e6-a38f5691e03a service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:14.326542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661eb79-1730-4e51-a4e6-a38f5691e03a service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:14.337337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11938a8-42e0-4a6a-a2a0-8bf8f765b84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:14.338615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11938a8-42e0-4a6a-a2a0-8bf8f765b84c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:14.340501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11938a8-42e0-4a6a-a2a0-8bf8f765b84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:14.340956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11938a8-42e0-4a6a-a2a0-8bf8f765b84c admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:14.341837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11938a8-42e0-4a6a-a2a0-8bf8f765b84c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:14.349046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11938a8-42e0-4a6a-a2a0-8bf8f765b84c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:14.452350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 826/1631] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:14 2022] POST /identity/v3/projects => generated 405 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:14.456757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95eac7-b831-4e92-a9ef-51d81862172f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:14.489971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a95eac7-b831-4e92-a9ef-51d81862172f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:14.775565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7932f0-4bed-4c60-8d97-a9de5ff18fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:14.776890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7932f0-4bed-4c60-8d97-a9de5ff18fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:14.777480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7932f0-4bed-4c60-8d97-a9de5ff18fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:14.778193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7932f0-4bed-4c60-8d97-a9de5ff18fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:14.779740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7932f0-4bed-4c60-8d97-a9de5ff18fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:14.783670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7932f0-4bed-4c60-8d97-a9de5ff18fc8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:14.856214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 806/1632] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:14 2022] POST /identity/v3/users => generated 309 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:14.860854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11938a8-42e0-4a6a-a2a0-8bf8f765b84c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:14.898596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11938a8-42e0-4a6a-a2a0-8bf8f765b84c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:14.919259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bfa7f-792e-4785-a987-4a8b1a4a3490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:14.922493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bfa7f-792e-4785-a987-4a8b1a4a3490 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:14.923168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bfa7f-792e-4785-a987-4a8b1a4a3490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:14.926397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4bfa7f-792e-4785-a987-4a8b1a4a3490 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:14.927870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bfa7f-792e-4785-a987-4a8b1a4a3490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:14.931351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4bfa7f-792e-4785-a987-4a8b1a4a3490 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:14.988477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7932f0-4bed-4c60-8d97-a9de5ff18fc8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:14.997781 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 827/1633] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:15.018578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7932f0-4bed-4c60-8d97-a9de5ff18fc8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:15.039930 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eb21f-163e-46f2-8409-38f1843f6e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:15.041124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb21f-163e-46f2-8409-38f1843f6e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:15.041610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb21f-163e-46f2-8409-38f1843f6e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:15.042136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102eb21f-163e-46f2-8409-38f1843f6e5d admin admin] PATH_INFO: `/v3/projects/0f930cf47e1942f6b2d372a51f019926/users/03e50f525b6e4be5936cea7836225fea/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:15.062250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:15.113820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102eb21f-163e-46f2-8409-38f1843f6e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f930cf47e1942f6b2d372a51f019926, user_id=03e50f525b6e4be5936cea7836225fea, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:15.120092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102eb21f-163e-46f2-8409-38f1843f6e5d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:15.190125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 807/1634] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:14 2022] PUT /identity/v3/projects/0f930cf47e1942f6b2d372a51f019926/users/03e50f525b6e4be5936cea7836225fea/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:15.196892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bfa7f-792e-4785-a987-4a8b1a4a3490 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:15.239833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4bfa7f-792e-4785-a987-4a8b1a4a3490 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:15.556093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3cd60-99f8-4418-af30-c2679853acde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:15.559861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3cd60-99f8-4418-af30-c2679853acde admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:15.560683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3cd60-99f8-4418-af30-c2679853acde admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:15.561462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3cd60-99f8-4418-af30-c2679853acde admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:15.566708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3cd60-99f8-4418-af30-c2679853acde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:15.573120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3cd60-99f8-4418-af30-c2679853acde admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:15.619309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 828/1635] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 424 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:15.625234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eb21f-163e-46f2-8409-38f1843f6e5d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:15.650631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102eb21f-163e-46f2-8409-38f1843f6e5d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:15.661362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c1dd1-f933-4728-8484-28067d488df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:15.662955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c1dd1-f933-4728-8484-28067d488df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:15.664319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c1dd1-f933-4728-8484-28067d488df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:15.664970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c1dd1-f933-4728-8484-28067d488df3 admin admin] PATH_INFO: `/v3/projects/0f930cf47e1942f6b2d372a51f019926/users/03e50f525b6e4be5936cea7836225fea/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:15.669592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c1dd1-f933-4728-8484-28067d488df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f930cf47e1942f6b2d372a51f019926, user_id=03e50f525b6e4be5936cea7836225fea, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:15.673528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c1dd1-f933-4728-8484-28067d488df3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:15.698934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 808/1636] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:15 2022] PUT /identity/v3/projects/0f930cf47e1942f6b2d372a51f019926/users/03e50f525b6e4be5936cea7836225fea/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:15.956034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3658ff-c8bf-4bc3-95ba-f3bcbcde51ce None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:15.956441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3658ff-c8bf-4bc3-95ba-f3bcbcde51ce None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:15.956829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3658ff-c8bf-4bc3-95ba-f3bcbcde51ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:15.997818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4b3658ff-c8bf-4bc3-95ba-f3bcbcde51ce None None] MFA Rules not processed for user `e8e345dbc2114b718c98c70844352c63`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:15.999369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4b3658ff-c8bf-4bc3-95ba-f3bcbcde51ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:16.158080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 829/1637] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:16.176294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aec22-d2bc-4bda-9dd3-c11dfb8a9e06 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:16.177255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aec22-d2bc-4bda-9dd3-c11dfb8a9e06 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:16.177742 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aec22-d2bc-4bda-9dd3-c11dfb8a9e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:16.218738 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7c5aec22-d2bc-4bda-9dd3-c11dfb8a9e06 None None] MFA Rules not processed for user `e8e345dbc2114b718c98c70844352c63`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:16.221881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7c5aec22-d2bc-4bda-9dd3-c11dfb8a9e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:16.249185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 809/1638] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:16.269963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3658ff-c8bf-4bc3-95ba-f3bcbcde51ce None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:16.403902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:16.475463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3658ff-c8bf-4bc3-95ba-f3bcbcde51ce None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:16.492455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fecf27-d359-4f0f-8e03-01797edb1c5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:16.493477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fecf27-d359-4f0f-8e03-01797edb1c5c service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:16.494481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fecf27-d359-4f0f-8e03-01797edb1c5c service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:16.494835 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fecf27-d359-4f0f-8e03-01797edb1c5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:16.508633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fecf27-d359-4f0f-8e03-01797edb1c5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:16.512271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fecf27-d359-4f0f-8e03-01797edb1c5c service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:16.550083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 830/1639] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:08:18.164743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aec22-d2bc-4bda-9dd3-c11dfb8a9e06 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:18.403075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aec22-d2bc-4bda-9dd3-c11dfb8a9e06 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:18.423602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:18.424470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:18.424876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:18.425253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] PATH_INFO: `/v3/users/629d7d6f348d4805bd10850d151daefb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:18.430869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=629d7d6f348d4805bd10850d151daefb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:18.435635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:18.458648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35916-32ec-4ff6-a865-953272297344 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:18.459146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35916-32ec-4ff6-a865-953272297344 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:18.459630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf35916-32ec-4ff6-a865-953272297344 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:18.465966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 831/1640] 10.222.0.16 () {54 vars in 912 bytes} [Fri Aug 19 21:08:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:08:18.479177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f3e3-5ab7-4475-8e10-b5034fd6bf14 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:18.479688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f3e3-5ab7-4475-8e10-b5034fd6bf14 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:18.480429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f3e3-5ab7-4475-8e10-b5034fd6bf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:18.632517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '629d7d6f348d4805bd10850d151daefb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:18.633230 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-81e1f3e3-5ab7-4475-8e10-b5034fd6bf14 None None] MFA Rules not processed for user `b37a4275e7c6441aa9e8bb0298f56589`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:18.635286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-81e1f3e3-5ab7-4475-8e10-b5034fd6bf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:18.673044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '629d7d6f348d4805bd10850d151daefb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:18.673967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '629d7d6f348d4805bd10850d151daefb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:18.676400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '629d7d6f348d4805bd10850d151daefb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:18.707416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '629d7d6f348d4805bd10850d151daefb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:18.774770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 832/1642] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:08:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:18.776811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e0903-ec8f-40d2-ac0e-62d3719cffde None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:18.777161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e0903-ec8f-40d2-ac0e-62d3719cffde None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:18.777436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e0903-ec8f-40d2-ac0e-62d3719cffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:18.777790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 810/1642] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:08:18 2022] DELETE /identity/v3/users/629d7d6f348d4805bd10850d151daefb => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:18.815932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:18.818406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:18.854648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bc1e0903-ec8f-40d2-ac0e-62d3719cffde None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:18.856372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bc1e0903-ec8f-40d2-ac0e-62d3719cffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:18.909621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:18.958835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25497fb4-8092-4106-96a8-be4751641067 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:18.973433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87e3fe-70c9-41c0-8a73-84f8bc8f6a99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:18.974551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87e3fe-70c9-41c0-8a73-84f8bc8f6a99 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:18.974835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87e3fe-70c9-41c0-8a73-84f8bc8f6a99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:18.975119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87e3fe-70c9-41c0-8a73-84f8bc8f6a99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:19.008551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87e3fe-70c9-41c0-8a73-84f8bc8f6a99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:19.010587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87e3fe-70c9-41c0-8a73-84f8bc8f6a99 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:19.022862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 833/1643] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:19.031936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 811/1644] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:19.202845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e0903-ec8f-40d2-ac0e-62d3719cffde None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:19.229184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e0903-ec8f-40d2-ac0e-62d3719cffde None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:19.250675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c703370-1e14-4f9d-b92f-bc08d46f3655 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:19.252223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c703370-1e14-4f9d-b92f-bc08d46f3655 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:19.252545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c703370-1e14-4f9d-b92f-bc08d46f3655 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:19.252828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c703370-1e14-4f9d-b92f-bc08d46f3655 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:19.294023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c703370-1e14-4f9d-b92f-bc08d46f3655 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:19.302435 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c703370-1e14-4f9d-b92f-bc08d46f3655 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:19.359856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 834/1645] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:21.119916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ce18-b194-4605-9d18-2a7542fc9340 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:21.120741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ce18-b194-4605-9d18-2a7542fc9340 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:21.121214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ce18-b194-4605-9d18-2a7542fc9340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:21.240737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5dd1ce18-b194-4605-9d18-2a7542fc9340 None None] MFA Rules not processed for user `03e50f525b6e4be5936cea7836225fea`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:21.250390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5dd1ce18-b194-4605-9d18-2a7542fc9340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:21.453252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e066c-0b1a-4a80-82ad-1550682c4d7d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:21.455621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e066c-0b1a-4a80-82ad-1550682c4d7d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:21.456563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e066c-0b1a-4a80-82ad-1550682c4d7d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:21.461378 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 835/1646] 10.222.0.16 () {54 vars in 961 bytes} [Fri Aug 19 21:08:21 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:08:21.476152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709b867-ba03-43b2-994a-8a846b0274a6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:21.477158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709b867-ba03-43b2-994a-8a846b0274a6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:21.478833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f709b867-ba03-43b2-994a-8a846b0274a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:21.480390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 812/1647] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:21 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:21.524793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f709b867-ba03-43b2-994a-8a846b0274a6 None None] MFA Rules not processed for user `824165e55d2f4742aa435f1edff9b403`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:21.528631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f709b867-ba03-43b2-994a-8a846b0274a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:21.665651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0f76-a38f-4789-b3db-52278cfe173f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:21.665941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0f76-a38f-4789-b3db-52278cfe173f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:21.666279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc0f76-a38f-4789-b3db-52278cfe173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:21.693647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:21.748310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-affc0f76-a38f-4789-b3db-52278cfe173f None None] MFA Rules not processed for user `e8e345dbc2114b718c98c70844352c63`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:21.751382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-affc0f76-a38f-4789-b3db-52278cfe173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:21.794432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 836/1648] 10.222.0.16 () {58 vars in 1058 bytes} [Fri Aug 19 21:08:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:21.801188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709b867-ba03-43b2-994a-8a846b0274a6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:21.836878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f709b867-ba03-43b2-994a-8a846b0274a6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:21.850097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156b4a5-d07c-4f06-b626-c98371b568a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:21.854603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156b4a5-d07c-4f06-b626-c98371b568a6 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:21.855470 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156b4a5-d07c-4f06-b626-c98371b568a6 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:21.857100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a156b4a5-d07c-4f06-b626-c98371b568a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:21.865131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a156b4a5-d07c-4f06-b626-c98371b568a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:21.962857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156b4a5-d07c-4f06-b626-c98371b568a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:21.968699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a156b4a5-d07c-4f06-b626-c98371b568a6 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:21.978460 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 813/1649] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:22.066444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 837/1650] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:22.226656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc0f76-a38f-4789-b3db-52278cfe173f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:22.230515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-affc0f76-a38f-4789-b3db-52278cfe173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:22.285082 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc0f76-a38f-4789-b3db-52278cfe173f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:22.299258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:22.300423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:22.302976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156b4a5-d07c-4f06-b626-c98371b568a6 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:22.306200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a156b4a5-d07c-4f06-b626-c98371b568a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:22.307961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:22.309373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] PATH_INFO: `/v3/users/65aaea0b67d94ad2a80a5acb7b4a3bec` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:22.313133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=65aaea0b67d94ad2a80a5acb7b4a3bec)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:22.318173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:22.359901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a156b4a5-d07c-4f06-b626-c98371b568a6 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:22.377455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:22.378415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:22.378806 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:22.378981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] PATH_INFO: `/v3/projects/6f7cafd832804f8bab8322ee27c1803d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:22.382131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f7cafd832804f8bab8322ee27c1803d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:22.390148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:22.501073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65aaea0b67d94ad2a80a5acb7b4a3bec'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:22.532434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65aaea0b67d94ad2a80a5acb7b4a3bec'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:22.533608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65aaea0b67d94ad2a80a5acb7b4a3bec'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:22.534833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65aaea0b67d94ad2a80a5acb7b4a3bec'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:22.566203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa06cd11-9ece-46d6-83a7-fdc651e369ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65aaea0b67d94ad2a80a5acb7b4a3bec'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:22.597924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:22.643924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 814/1651] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:08:22 2022] DELETE /identity/v3/users/65aaea0b67d94ad2a80a5acb7b4a3bec => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:22.651882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a4a1a-6014-4e54-8d48-88ce86d3467e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:22.653489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a4a1a-6014-4e54-8d48-88ce86d3467e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:22.657622 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a4a1a-6014-4e54-8d48-88ce86d3467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:22.698870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f7cafd832804f8bab8322ee27c1803d', 'request_id': 'req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:22.718063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f7cafd832804f8bab8322ee27c1803d', 'request_id': 'req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:22.733948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f7cafd832804f8bab8322ee27c1803d', 'request_id': 'req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:22.736911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4c6a4a1a-6014-4e54-8d48-88ce86d3467e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:22.739723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c6a4a1a-6014-4e54-8d48-88ce86d3467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:22.797950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] The token cache is being invalidate because project 6f7cafd832804f8bab8322ee27c1803d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:08:22.798256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:22.799592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cdc76c6c-5ae0-4e84-b9cc-208d9b06411e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:22.806073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 838/1652] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:08:22 2022] DELETE /identity/v3/projects/6f7cafd832804f8bab8322ee27c1803d => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:22.813481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92ce58-584a-44d6-adb0-412ef85bd0ab None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:22.813801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92ce58-584a-44d6-adb0-412ef85bd0ab None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:22.814076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92ce58-584a-44d6-adb0-412ef85bd0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:22.894874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ef92ce58-584a-44d6-adb0-412ef85bd0ab None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:22.903609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ef92ce58-584a-44d6-adb0-412ef85bd0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:22.941857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 815/1653] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:23.074375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a4a1a-6014-4e54-8d48-88ce86d3467e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:23.076077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c6a4a1a-6014-4e54-8d48-88ce86d3467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:23.119092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 839/1654] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:23.121212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a2b4-b886-49fb-8bbb-c972f40d74d4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.121730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a2b4-b886-49fb-8bbb-c972f40d74d4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.121971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0a2b4-b886-49fb-8bbb-c972f40d74d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:23.166404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9bb0a2b4-b886-49fb-8bbb-c972f40d74d4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:23.168459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9bb0a2b4-b886-49fb-8bbb-c972f40d74d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:23.196972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 840/1655] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:23.203905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0a2b4-b886-49fb-8bbb-c972f40d74d4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:23.245666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0a2b4-b886-49fb-8bbb-c972f40d74d4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:23.260548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e79ca-3cc0-478f-96e3-3704b566a106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:23.261229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e79ca-3cc0-478f-96e3-3704b566a106 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.261512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e79ca-3cc0-478f-96e3-3704b566a106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.261866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e79ca-3cc0-478f-96e3-3704b566a106 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:23.262549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e79ca-3cc0-478f-96e3-3704b566a106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:23.262859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e79ca-3cc0-478f-96e3-3704b566a106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:23.264731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e79ca-3cc0-478f-96e3-3704b566a106 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:23.302245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 841/1656] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:08:23.308988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e79ca-3cc0-478f-96e3-3704b566a106 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:23.327961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e79ca-3cc0-478f-96e3-3704b566a106 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:23.337147 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a4a1a-6014-4e54-8d48-88ce86d3467e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:23.342918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a955e-7e5b-4f74-906c-db20ea957a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:23.343569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a955e-7e5b-4f74-906c-db20ea957a30 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.343852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a955e-7e5b-4f74-906c-db20ea957a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.344283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a955e-7e5b-4f74-906c-db20ea957a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:23.344944 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a955e-7e5b-4f74-906c-db20ea957a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:23.346994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a955e-7e5b-4f74-906c-db20ea957a30 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:23.355993 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f66a201-9506-4170-9bb8-4c8b46a8c84c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:23.357359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66a201-9506-4170-9bb8-4c8b46a8c84c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.358200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66a201-9506-4170-9bb8-4c8b46a8c84c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.359211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66a201-9506-4170-9bb8-4c8b46a8c84c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:23.396080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f66a201-9506-4170-9bb8-4c8b46a8c84c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:23.400862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f66a201-9506-4170-9bb8-4c8b46a8c84c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:23.427984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 842/1657] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:23 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:23.430307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a955e-7e5b-4f74-906c-db20ea957a30 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:23.449685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 816/1658] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:08:23.485489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:23.496717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143c392-b54e-4a3d-83a0-6aff8ae0af3d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.497375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143c392-b54e-4a3d-83a0-6aff8ae0af3d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.498019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143c392-b54e-4a3d-83a0-6aff8ae0af3d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:23.501601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 817/1659] 10.222.0.16 () {54 vars in 912 bytes} [Fri Aug 19 21:08:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 19 21:08:23.517494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9f15b-3f77-4481-8d31-ddf415d35ec7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.518181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9f15b-3f77-4481-8d31-ddf415d35ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.518740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9f15b-3f77-4481-8d31-ddf415d35ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:23.607700 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6cf9f15b-3f77-4481-8d31-ddf415d35ec7 None None] MFA Rules not processed for user `b37a4275e7c6441aa9e8bb0298f56589`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:23.608223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a955e-7e5b-4f74-906c-db20ea957a30 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:23.610219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6cf9f15b-3f77-4481-8d31-ddf415d35ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:23.631388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fcf6e-66b5-4f1f-bce2-427fb2a2559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:23.635568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fcf6e-66b5-4f1f-bce2-427fb2a2559c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.636082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fcf6e-66b5-4f1f-bce2-427fb2a2559c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.637285 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fcf6e-66b5-4f1f-bce2-427fb2a2559c admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:23.639130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fcf6e-66b5-4f1f-bce2-427fb2a2559c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:23.643691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fcf6e-66b5-4f1f-bce2-427fb2a2559c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:23.694841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 843/1660] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:23 2022] POST /identity/v3/users => generated 286 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:08:23.700592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fcf6e-66b5-4f1f-bce2-427fb2a2559c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:23.723312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fcf6e-66b5-4f1f-bce2-427fb2a2559c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:23.734769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a3c17-fcb0-4475-bbfe-182ef1838013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:23.735862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a3c17-fcb0-4475-bbfe-182ef1838013 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.736204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a3c17-fcb0-4475-bbfe-182ef1838013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.736468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1a3c17-fcb0-4475-bbfe-182ef1838013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:23.738093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1a3c17-fcb0-4475-bbfe-182ef1838013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:23.744345 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1a3c17-fcb0-4475-bbfe-182ef1838013 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:23.771316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 844/1661] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:23.776681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a3c17-fcb0-4475-bbfe-182ef1838013 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:23.798054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1a3c17-fcb0-4475-bbfe-182ef1838013 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:23.811120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67fff3-fe89-44f7-9c62-1c212225ec99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:23.811851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67fff3-fe89-44f7-9c62-1c212225ec99 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.812132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67fff3-fe89-44f7-9c62-1c212225ec99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.812406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67fff3-fe89-44f7-9c62-1c212225ec99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:23.815394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67fff3-fe89-44f7-9c62-1c212225ec99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:23.820166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b67fff3-fe89-44f7-9c62-1c212225ec99 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:23.845711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 845/1662] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:08:23.851843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67fff3-fe89-44f7-9c62-1c212225ec99 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:23.874096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67fff3-fe89-44f7-9c62-1c212225ec99 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:23.886058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690868a6-b4a6-408b-a7eb-1ebb04a1ae43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:23.886811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690868a6-b4a6-408b-a7eb-1ebb04a1ae43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.887115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690868a6-b4a6-408b-a7eb-1ebb04a1ae43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.887357 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690868a6-b4a6-408b-a7eb-1ebb04a1ae43 admin admin] PATH_INFO: `/v3/projects/70f9c2b1ba304396b0908bf122467da3/users/a50e2738b9d541e0bb94948d6fbe0895/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:23.898137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 818/1663] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:08:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 19 21:08:23.926866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690868a6-b4a6-408b-a7eb-1ebb04a1ae43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70f9c2b1ba304396b0908bf122467da3, user_id=a50e2738b9d541e0bb94948d6fbe0895, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:23.938725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690868a6-b4a6-408b-a7eb-1ebb04a1ae43 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:23.955224 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9f15b-3f77-4481-8d31-ddf415d35ec7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:23.977823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 846/1664] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:23 2022] PUT /identity/v3/projects/70f9c2b1ba304396b0908bf122467da3/users/a50e2738b9d541e0bb94948d6fbe0895/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:08:23.988514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45084f14-45b8-44fb-b514-063558d09fe6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:23.989933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45084f14-45b8-44fb-b514-063558d09fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:23.989933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45084f14-45b8-44fb-b514-063558d09fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:24.006122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9f15b-3f77-4481-8d31-ddf415d35ec7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:24.050083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-45084f14-45b8-44fb-b514-063558d09fe6 None None] MFA Rules not processed for user `a50e2738b9d541e0bb94948d6fbe0895`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:24.051387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-45084f14-45b8-44fb-b514-063558d09fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:24.221256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 847/1665] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:24.222997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45084f14-45b8-44fb-b514-063558d09fe6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:24.239084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:24.301798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917e121-2c02-4790-a0df-ed4cabe1ae03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:24.302819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917e121-2c02-4790-a0df-ed4cabe1ae03 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:24.304056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917e121-2c02-4790-a0df-ed4cabe1ae03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:24.305012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917e121-2c02-4790-a0df-ed4cabe1ae03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:24.325584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917e121-2c02-4790-a0df-ed4cabe1ae03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:24.329462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917e121-2c02-4790-a0df-ed4cabe1ae03 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:24.350525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 819/1666] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:24.381274 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:24.406473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45084f14-45b8-44fb-b514-063558d09fe6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:24.419150 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4825834-59b7-4b27-9160-0f324104836e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:24.419824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4825834-59b7-4b27-9160-0f324104836e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:24.420172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4825834-59b7-4b27-9160-0f324104836e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:24.420459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4825834-59b7-4b27-9160-0f324104836e admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:24.421147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4825834-59b7-4b27-9160-0f324104836e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:24.423179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4825834-59b7-4b27-9160-0f324104836e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:24.499106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 848/1667] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:24 2022] POST /identity/v3/projects => generated 371 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:24.506847 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917e121-2c02-4790-a0df-ed4cabe1ae03 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:24.591978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917e121-2c02-4790-a0df-ed4cabe1ae03 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:24.860880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cd133-af19-483d-a5fa-2aef323e0086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:24.861507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cd133-af19-483d-a5fa-2aef323e0086 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:24.863028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cd133-af19-483d-a5fa-2aef323e0086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:24.863354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cd133-af19-483d-a5fa-2aef323e0086 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:24.865196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cd133-af19-483d-a5fa-2aef323e0086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:24.874288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cd133-af19-483d-a5fa-2aef323e0086 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:24.914934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 820/1668] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:24 2022] POST /identity/v3/users => generated 292 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:24.919716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4825834-59b7-4b27-9160-0f324104836e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:24.949374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4825834-59b7-4b27-9160-0f324104836e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:24.962467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b1d40-2031-4522-bbda-0082cdb545e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:24.963068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1d40-2031-4522-bbda-0082cdb545e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:24.963337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1d40-2031-4522-bbda-0082cdb545e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:24.963611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413b1d40-2031-4522-bbda-0082cdb545e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:24.964291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b1d40-2031-4522-bbda-0082cdb545e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:24.966063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413b1d40-2031-4522-bbda-0082cdb545e8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:25.006458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 849/1669] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:25.008832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cd133-af19-483d-a5fa-2aef323e0086 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:25.097200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cd133-af19-483d-a5fa-2aef323e0086 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:25.147579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def26783-8f5a-4e4e-9b8f-9b73ba823ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:25.149115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def26783-8f5a-4e4e-9b8f-9b73ba823ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:25.158271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def26783-8f5a-4e4e-9b8f-9b73ba823ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:25.159728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def26783-8f5a-4e4e-9b8f-9b73ba823ae5 admin admin] PATH_INFO: `/v3/projects/de1fea58ee9c42e4bdaefd2c6e6984ea/users/351d9c59a837484ab70dbe0e54a25e5e/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:25.253987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def26783-8f5a-4e4e-9b8f-9b73ba823ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1fea58ee9c42e4bdaefd2c6e6984ea, user_id=351d9c59a837484ab70dbe0e54a25e5e, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:25.277263 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def26783-8f5a-4e4e-9b8f-9b73ba823ae5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:25.365884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 821/1670] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:25 2022] PUT /identity/v3/projects/de1fea58ee9c42e4bdaefd2c6e6984ea/users/351d9c59a837484ab70dbe0e54a25e5e/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:25.383536 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b1d40-2031-4522-bbda-0082cdb545e8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:25.403936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413b1d40-2031-4522-bbda-0082cdb545e8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:25.539766 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7aaf5f-8c82-4859-98e2-b530aa1d62df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:25.540734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7aaf5f-8c82-4859-98e2-b530aa1d62df admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:25.541187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7aaf5f-8c82-4859-98e2-b530aa1d62df admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:25.543442 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7aaf5f-8c82-4859-98e2-b530aa1d62df admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:25.544399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7aaf5f-8c82-4859-98e2-b530aa1d62df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:25.550606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7aaf5f-8c82-4859-98e2-b530aa1d62df admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:25.582018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def26783-8f5a-4e4e-9b8f-9b73ba823ae5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:25.589206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 850/1671] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:25.597718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def26783-8f5a-4e4e-9b8f-9b73ba823ae5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:25.606863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12d94f-9ce4-4caf-a365-69e22969ec0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:25.607958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12d94f-9ce4-4caf-a365-69e22969ec0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:25.608490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12d94f-9ce4-4caf-a365-69e22969ec0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:25.608917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12d94f-9ce4-4caf-a365-69e22969ec0b admin admin] PATH_INFO: `/v3/projects/de1fea58ee9c42e4bdaefd2c6e6984ea/users/351d9c59a837484ab70dbe0e54a25e5e/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:25.612384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12d94f-9ce4-4caf-a365-69e22969ec0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1fea58ee9c42e4bdaefd2c6e6984ea, user_id=351d9c59a837484ab70dbe0e54a25e5e, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:25.615967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12d94f-9ce4-4caf-a365-69e22969ec0b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:25.646961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 822/1672] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:25 2022] PUT /identity/v3/projects/de1fea58ee9c42e4bdaefd2c6e6984ea/users/351d9c59a837484ab70dbe0e54a25e5e/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:08:25.660924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98466f-9b76-4354-91e0-2691615c0222 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:25.661746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98466f-9b76-4354-91e0-2691615c0222 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:25.662660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98466f-9b76-4354-91e0-2691615c0222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:25.782152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd79cc-9f4d-47f6-a61c-9d497f152f8b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:25.782846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd79cc-9f4d-47f6-a61c-9d497f152f8b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:25.783199 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd79cc-9f4d-47f6-a61c-9d497f152f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:25.797855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8d98466f-9b76-4354-91e0-2691615c0222 None None] MFA Rules not processed for user `351d9c59a837484ab70dbe0e54a25e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:25.800715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8d98466f-9b76-4354-91e0-2691615c0222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:25.903463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-28fd79cc-9f4d-47f6-a61c-9d497f152f8b None None] MFA Rules not processed for user `593a0461ca714cff98ae43eb8833197c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:25.904984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-28fd79cc-9f4d-47f6-a61c-9d497f152f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:26.036978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 823/1673] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:26.050094 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d74984-6c1e-4ade-976b-99e62c58b8b0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:26.050700 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d74984-6c1e-4ade-976b-99e62c58b8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:26.051160 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d74984-6c1e-4ade-976b-99e62c58b8b0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:26.054627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 824/1674] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:08:26 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:08:26.055374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8ed7e-afc8-4f71-a681-91a9c8268b80 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:26.056627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8ed7e-afc8-4f71-a681-91a9c8268b80 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:26.056627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8ed7e-afc8-4f71-a681-91a9c8268b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:26.086720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d8b8ed7e-afc8-4f71-a681-91a9c8268b80 None None] MFA Rules not processed for user `593a0461ca714cff98ae43eb8833197c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:26.086720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d8b8ed7e-afc8-4f71-a681-91a9c8268b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:26.099493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 825/1675] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:26.119487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8ed7e-afc8-4f71-a681-91a9c8268b80 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:26.122985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d8b8ed7e-afc8-4f71-a681-91a9c8268b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:26.171277 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 851/1676] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:26.190694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42963941-c587-4ca1-b7dc-689023988b29 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:26.191017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42963941-c587-4ca1-b7dc-689023988b29 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:26.191279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42963941-c587-4ca1-b7dc-689023988b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:26.249829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:26.251287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-42963941-c587-4ca1-b7dc-689023988b29 None None] MFA Rules not processed for user `824165e55d2f4742aa435f1edff9b403`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:26.252934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-42963941-c587-4ca1-b7dc-689023988b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:26.395112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:26.419211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8ed7e-afc8-4f71-a681-91a9c8268b80 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:26.447130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84096e-456c-46fa-b783-a22ebb650ee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:26.447993 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84096e-456c-46fa-b783-a22ebb650ee1 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:26.448310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84096e-456c-46fa-b783-a22ebb650ee1 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:26.448598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84096e-456c-46fa-b783-a22ebb650ee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:26.453649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 852/1677] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:08:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:26.454371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42963941-c587-4ca1-b7dc-689023988b29 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:26.459692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84096e-456c-46fa-b783-a22ebb650ee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:26.461439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84096e-456c-46fa-b783-a22ebb650ee1 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:26.463995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-42963941-c587-4ca1-b7dc-689023988b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:26.493650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 826/1678] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:26.642439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84096e-456c-46fa-b783-a22ebb650ee1 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:26.644856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cb84096e-456c-46fa-b783-a22ebb650ee1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:26.672943 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84096e-456c-46fa-b783-a22ebb650ee1 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:26.681501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49d9e4-8212-4fb6-ab86-5df01fe4b15b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:26.682769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49d9e4-8212-4fb6-ab86-5df01fe4b15b service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:26.683296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49d9e4-8212-4fb6-ab86-5df01fe4b15b service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:26.684043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49d9e4-8212-4fb6-ab86-5df01fe4b15b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:26.686502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42963941-c587-4ca1-b7dc-689023988b29 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:26.698342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d5a66c-f693-4a3c-8ce1-50d11d413e0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:26.698998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5a66c-f693-4a3c-8ce1-50d11d413e0f service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:26.699286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5a66c-f693-4a3c-8ce1-50d11d413e0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:26.699585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d5a66c-f693-4a3c-8ce1-50d11d413e0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:26.722268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49d9e4-8212-4fb6-ab86-5df01fe4b15b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:26.729736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49d9e4-8212-4fb6-ab86-5df01fe4b15b service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:26.734658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5a66c-f693-4a3c-8ce1-50d11d413e0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:26.740322 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d5a66c-f693-4a3c-8ce1-50d11d413e0f service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:26.777348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 827/1679] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:08:26.787138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 853/1680] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:26 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:26.975251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49d9e4-8212-4fb6-ab86-5df01fe4b15b service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:27.013204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49d9e4-8212-4fb6-ab86-5df01fe4b15b service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:27.027789 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91387c-d408-409c-a790-f7d2247b32a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:27.028731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91387c-d408-409c-a790-f7d2247b32a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:27.029953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91387c-d408-409c-a790-f7d2247b32a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:27.029953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91387c-d408-409c-a790-f7d2247b32a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:27.111437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef9997-d153-4255-a22e-65a248dc16d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:27.112033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef9997-d153-4255-a22e-65a248dc16d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:27.112386 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aef9997-d153-4255-a22e-65a248dc16d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:27.234521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7aef9997-d153-4255-a22e-65a248dc16d5 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:27.235942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7aef9997-d153-4255-a22e-65a248dc16d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:27.263686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91387c-d408-409c-a790-f7d2247b32a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:27.265735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91387c-d408-409c-a790-f7d2247b32a8 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:27.291882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 828/1681] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:27.387994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 854/1682] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:08:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:27.615078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91387c-d408-409c-a790-f7d2247b32a8 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:27.618132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8a91387c-d408-409c-a790-f7d2247b32a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:27.710171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef9997-d153-4255-a22e-65a248dc16d5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:27.725107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7aef9997-d153-4255-a22e-65a248dc16d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:27.916744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:28.011743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91387c-d408-409c-a790-f7d2247b32a8 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:28.012073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aef9997-d153-4255-a22e-65a248dc16d5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:28.028484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7948dfa-5807-424a-8b6f-492350365c03 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:28.028484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7948dfa-5807-424a-8b6f-492350365c03 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:28.028484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7948dfa-5807-424a-8b6f-492350365c03 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:28.028484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7948dfa-5807-424a-8b6f-492350365c03 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:28.039216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:28.040366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:28.040881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:28.041362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] PATH_INFO: `/v3/projects/c654ccea4cb549a5848505a75432b760` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:28.045352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c654ccea4cb549a5848505a75432b760)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:28.054602 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7948dfa-5807-424a-8b6f-492350365c03 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:28.055284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:08:28.055284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Aug 19 21:08:28.055284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: Aug 19 21:08:28.056484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7948dfa-5807-424a-8b6f-492350365c03 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:28.057856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:28.068013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 855/1683] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:08:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:08:28.398955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:28.472704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c654ccea4cb549a5848505a75432b760', 'request_id': 'req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:28.527463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c654ccea4cb549a5848505a75432b760', 'request_id': 'req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:28.551324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c654ccea4cb549a5848505a75432b760', 'request_id': 'req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:28.593740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] The token cache is being invalidate because project c654ccea4cb549a5848505a75432b760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:08:28.594534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:28.599117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-14c5b39b-62c1-4e06-a0c5-5f3086e200e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:28.606509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 829/1684] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:08:27 2022] DELETE /identity/v3/projects/c654ccea4cb549a5848505a75432b760 => generated 0 bytes in 993 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:28.611869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420b86b-4c40-4e61-b04e-86cfedfb58e6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:28.612800 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420b86b-4c40-4e61-b04e-86cfedfb58e6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:28.613394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420b86b-4c40-4e61-b04e-86cfedfb58e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:28.670494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0420b86b-4c40-4e61-b04e-86cfedfb58e6 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:28.672723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0420b86b-4c40-4e61-b04e-86cfedfb58e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:29.184647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 856/1685] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 574 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:29.193021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d351529-57a7-40ec-9fb7-08204d447d2f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:29.193533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d351529-57a7-40ec-9fb7-08204d447d2f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:29.194840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d351529-57a7-40ec-9fb7-08204d447d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:29.376259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0d351529-57a7-40ec-9fb7-08204d447d2f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:29.420296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0d351529-57a7-40ec-9fb7-08204d447d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:29.506355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420b86b-4c40-4e61-b04e-86cfedfb58e6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:29.509929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 830/1686] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:29.527073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0420b86b-4c40-4e61-b04e-86cfedfb58e6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:29.566233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1052460-3995-47b6-8295-f99b9c8f9d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:29.567228 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1052460-3995-47b6-8295-f99b9c8f9d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:29.568395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1052460-3995-47b6-8295-f99b9c8f9d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:29.568395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1052460-3995-47b6-8295-f99b9c8f9d21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:29.568587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1052460-3995-47b6-8295-f99b9c8f9d21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:29.568879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1052460-3995-47b6-8295-f99b9c8f9d21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:29.571368 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1052460-3995-47b6-8295-f99b9c8f9d21 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:29.671442 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 857/1687] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:08:29.674211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d351529-57a7-40ec-9fb7-08204d447d2f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:29.718299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d351529-57a7-40ec-9fb7-08204d447d2f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:29.738097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997930dc-8b89-4ac0-910a-2d8a1349b5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:29.738205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997930dc-8b89-4ac0-910a-2d8a1349b5cf admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:29.738508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997930dc-8b89-4ac0-910a-2d8a1349b5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:29.738731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997930dc-8b89-4ac0-910a-2d8a1349b5cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:29.739411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997930dc-8b89-4ac0-910a-2d8a1349b5cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:29.741532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997930dc-8b89-4ac0-910a-2d8a1349b5cf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:29.878214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 831/1688] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:29 2022] POST /identity/v3/projects => generated 381 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:29.879491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1052460-3995-47b6-8295-f99b9c8f9d21 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:29.905243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1052460-3995-47b6-8295-f99b9c8f9d21 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:30.207355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744a67-62f5-4490-9d3c-44ab1ae5c9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:30.210565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744a67-62f5-4490-9d3c-44ab1ae5c9ca admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:30.213799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744a67-62f5-4490-9d3c-44ab1ae5c9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:30.216334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3744a67-62f5-4490-9d3c-44ab1ae5c9ca admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:30.221273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3744a67-62f5-4490-9d3c-44ab1ae5c9ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:30.227786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3744a67-62f5-4490-9d3c-44ab1ae5c9ca admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:30.428558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997930dc-8b89-4ac0-910a-2d8a1349b5cf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:30.430775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 858/1689] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:29 2022] POST /identity/v3/users => generated 291 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:08:30.480562 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997930dc-8b89-4ac0-910a-2d8a1349b5cf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:30.550716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6a591-cef4-4f47-b507-6a8aa4a965f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:30.552256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6a591-cef4-4f47-b507-6a8aa4a965f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:30.553045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6a591-cef4-4f47-b507-6a8aa4a965f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:30.559798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6a591-cef4-4f47-b507-6a8aa4a965f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:30.621550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6a591-cef4-4f47-b507-6a8aa4a965f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:30.639966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6a591-cef4-4f47-b507-6a8aa4a965f4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:30.697065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744a67-62f5-4490-9d3c-44ab1ae5c9ca admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:30.727301 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 832/1690] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:08:30.769782 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3744a67-62f5-4490-9d3c-44ab1ae5c9ca admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:30.818177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928eced-27c7-4ed9-ab0e-d8c9c91db961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:30.820399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928eced-27c7-4ed9-ab0e-d8c9c91db961 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:30.821732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928eced-27c7-4ed9-ab0e-d8c9c91db961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:30.826653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928eced-27c7-4ed9-ab0e-d8c9c91db961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:30.827839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928eced-27c7-4ed9-ab0e-d8c9c91db961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:30.831104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928eced-27c7-4ed9-ab0e-d8c9c91db961 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:30.942934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6a591-cef4-4f47-b507-6a8aa4a965f4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:30.977161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 859/1691] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:31.018540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6a591-cef4-4f47-b507-6a8aa4a965f4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:31.049581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48567dd-96e1-4c9f-8004-d8329547807a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:31.051314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48567dd-96e1-4c9f-8004-d8329547807a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:31.052164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48567dd-96e1-4c9f-8004-d8329547807a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:31.053008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48567dd-96e1-4c9f-8004-d8329547807a admin admin] PATH_INFO: `/v3/projects/6194aa367f994e3e8f90baa7e5d3c97b/users/750b61462478471fa4e1e2f45c269109/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:31.169102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48567dd-96e1-4c9f-8004-d8329547807a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6194aa367f994e3e8f90baa7e5d3c97b, user_id=750b61462478471fa4e1e2f45c269109, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:31.182425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48567dd-96e1-4c9f-8004-d8329547807a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:31.259288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 833/1692] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:30 2022] PUT /identity/v3/projects/6194aa367f994e3e8f90baa7e5d3c97b/users/750b61462478471fa4e1e2f45c269109/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:31.269717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b1a9-b9d9-4e46-a376-27399daa5d31 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:31.270003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b1a9-b9d9-4e46-a376-27399daa5d31 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:31.270284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b1a9-b9d9-4e46-a376-27399daa5d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:31.423466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9ad8b1a9-b9d9-4e46-a376-27399daa5d31 None None] MFA Rules not processed for user `750b61462478471fa4e1e2f45c269109`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:31.425986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9ad8b1a9-b9d9-4e46-a376-27399daa5d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:31.646354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:31.981286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48567dd-96e1-4c9f-8004-d8329547807a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:31.994303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 860/1693] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 718 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:32.363573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48567dd-96e1-4c9f-8004-d8329547807a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:32.374161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d91a1-14a0-4150-896f-cac1096024d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:32.378176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d91a1-14a0-4150-896f-cac1096024d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:32.379044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d91a1-14a0-4150-896f-cac1096024d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:32.379494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d91a1-14a0-4150-896f-cac1096024d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:32.381613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d91a1-14a0-4150-896f-cac1096024d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:32.385425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d91a1-14a0-4150-896f-cac1096024d2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:32.482766 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 834/1694] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:31 2022] POST /identity/v3/projects => generated 381 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:32.489190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b1a9-b9d9-4e46-a376-27399daa5d31 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:32.543787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b1a9-b9d9-4e46-a376-27399daa5d31 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:32.801336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec365ae-ccb0-4bb3-b709-03e375885525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:32.805904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec365ae-ccb0-4bb3-b709-03e375885525 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:32.806415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec365ae-ccb0-4bb3-b709-03e375885525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:32.806844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec365ae-ccb0-4bb3-b709-03e375885525 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:32.807717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec365ae-ccb0-4bb3-b709-03e375885525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:32.810706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec365ae-ccb0-4bb3-b709-03e375885525 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:32.856537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 861/1695] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:32 2022] POST /identity/v3/users => generated 297 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:32.862552 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d91a1-14a0-4150-896f-cac1096024d2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:32.895355 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d91a1-14a0-4150-896f-cac1096024d2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:32.946189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744362-3dd2-4017-9255-3981e96f98d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:32.950152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744362-3dd2-4017-9255-3981e96f98d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:32.951196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744362-3dd2-4017-9255-3981e96f98d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:32.951926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744362-3dd2-4017-9255-3981e96f98d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:32.955316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f744362-3dd2-4017-9255-3981e96f98d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:32.961484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f744362-3dd2-4017-9255-3981e96f98d4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:33.030424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 835/1696] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:33.034475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec365ae-ccb0-4bb3-b709-03e375885525 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:33.052629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec365ae-ccb0-4bb3-b709-03e375885525 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:33.069742 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b77cc0-cdae-4c5a-9c72-c033700aee58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:33.070518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b77cc0-cdae-4c5a-9c72-c033700aee58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:33.070930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b77cc0-cdae-4c5a-9c72-c033700aee58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:33.071214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b77cc0-cdae-4c5a-9c72-c033700aee58 admin admin] PATH_INFO: `/v3/projects/8bdf0a4289c846ca9f5c87bc334fd9d6/users/97099ad0c973491a8fe24f2167ec4b08/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:33.136076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b77cc0-cdae-4c5a-9c72-c033700aee58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdf0a4289c846ca9f5c87bc334fd9d6, user_id=97099ad0c973491a8fe24f2167ec4b08, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:33.139226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b77cc0-cdae-4c5a-9c72-c033700aee58 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:33.174838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 862/1697] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:33 2022] PUT /identity/v3/projects/8bdf0a4289c846ca9f5c87bc334fd9d6/users/97099ad0c973491a8fe24f2167ec4b08/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:33.184095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744362-3dd2-4017-9255-3981e96f98d4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:33.208996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744362-3dd2-4017-9255-3981e96f98d4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:33.375039 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:33.501599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98990e5a-085f-400b-80c7-7ff6fdbde3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:33.516253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98990e5a-085f-400b-80c7-7ff6fdbde3e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:33.517284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98990e5a-085f-400b-80c7-7ff6fdbde3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:33.517959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98990e5a-085f-400b-80c7-7ff6fdbde3e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:33.523627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98990e5a-085f-400b-80c7-7ff6fdbde3e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:33.525876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98990e5a-085f-400b-80c7-7ff6fdbde3e4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:33.562746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 836/1698] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:33.564073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b77cc0-cdae-4c5a-9c72-c033700aee58 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:33.602984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b77cc0-cdae-4c5a-9c72-c033700aee58 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:33.619958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85ccd2-e0b0-4bf8-9cba-d73788440ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:33.622007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85ccd2-e0b0-4bf8-9cba-d73788440ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:33.622819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85ccd2-e0b0-4bf8-9cba-d73788440ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:33.623566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85ccd2-e0b0-4bf8-9cba-d73788440ca2 admin admin] PATH_INFO: `/v3/projects/8bdf0a4289c846ca9f5c87bc334fd9d6/users/97099ad0c973491a8fe24f2167ec4b08/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:33.628254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85ccd2-e0b0-4bf8-9cba-d73788440ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdf0a4289c846ca9f5c87bc334fd9d6, user_id=97099ad0c973491a8fe24f2167ec4b08, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:33.635305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85ccd2-e0b0-4bf8-9cba-d73788440ca2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:33.679429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 863/1699] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:33 2022] PUT /identity/v3/projects/8bdf0a4289c846ca9f5c87bc334fd9d6/users/97099ad0c973491a8fe24f2167ec4b08/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:33.693143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495573a-785a-49f6-ab6f-2576365aefeb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:33.693518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495573a-785a-49f6-ab6f-2576365aefeb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:33.694076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495573a-785a-49f6-ab6f-2576365aefeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:33.768282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8495573a-785a-49f6-ab6f-2576365aefeb None None] MFA Rules not processed for user `97099ad0c973491a8fe24f2167ec4b08`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:33.770870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8495573a-785a-49f6-ab6f-2576365aefeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:33.885757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 837/1700] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:33.910783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85ccd2-e0b0-4bf8-9cba-d73788440ca2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:33.914101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4e85ccd2-e0b0-4bf8-9cba-d73788440ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:34.253007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85ccd2-e0b0-4bf8-9cba-d73788440ca2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:34.270383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3de72-03fc-4c29-abad-b4600b4cdb63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:34.271556 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3de72-03fc-4c29-abad-b4600b4cdb63 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:34.272092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3de72-03fc-4c29-abad-b4600b4cdb63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:34.280066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3de72-03fc-4c29-abad-b4600b4cdb63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:34.329727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3de72-03fc-4c29-abad-b4600b4cdb63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:34.333116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3de72-03fc-4c29-abad-b4600b4cdb63 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:34.423421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 864/1701] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:34.560638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5789f-db27-4594-9c7c-c7c73b9bace2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:34.562520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5789f-db27-4594-9c7c-c7c73b9bace2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:34.563779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e5789f-db27-4594-9c7c-c7c73b9bace2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:34.570761 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 838/1702] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:08:34 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:08:34.573630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3de72-03fc-4c29-abad-b4600b4cdb63 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:34.586094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3de72-03fc-4c29-abad-b4600b4cdb63 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:34.599227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19844b-3b04-4555-8752-58c01156b123 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:34.600098 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19844b-3b04-4555-8752-58c01156b123 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:34.600098 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19844b-3b04-4555-8752-58c01156b123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:34.606920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db008ff9-ef39-4e77-850a-943d29101eb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:34.606920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db008ff9-ef39-4e77-850a-943d29101eb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:34.607381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db008ff9-ef39-4e77-850a-943d29101eb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:34.607381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db008ff9-ef39-4e77-850a-943d29101eb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:34.661865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:34.676920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4b19844b-3b04-4555-8752-58c01156b123 None None] MFA Rules not processed for user `824165e55d2f4742aa435f1edff9b403`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:34.683803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4b19844b-3b04-4555-8752-58c01156b123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:34.811177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db008ff9-ef39-4e77-850a-943d29101eb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:34.812593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db008ff9-ef39-4e77-850a-943d29101eb3 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:34.852681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 865/1703] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:34.869303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f52a7d-bed5-4a4b-b6dc-a007b088b27c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:34.869593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f52a7d-bed5-4a4b-b6dc-a007b088b27c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:34.869906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f52a7d-bed5-4a4b-b6dc-a007b088b27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:34.987161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 839/1704] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:34.988402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19844b-3b04-4555-8752-58c01156b123 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:34.990144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4b19844b-3b04-4555-8752-58c01156b123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:34.993626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-21f52a7d-bed5-4a4b-b6dc-a007b088b27c None None] MFA Rules not processed for user `7f46019e78cc4e89bf877a4f5a7a8fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:34.996107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-21f52a7d-bed5-4a4b-b6dc-a007b088b27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:35.073621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:35.110307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19844b-3b04-4555-8752-58c01156b123 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:35.118551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:35.119626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:35.120119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:35.120602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] PATH_INFO: `/v3/users/4c338e58dc874c61b78364907397a85b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:35.123652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c338e58dc874c61b78364907397a85b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:35.167333 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:35.313439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 866/1705] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:35.316596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f52a7d-bed5-4a4b-b6dc-a007b088b27c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:35.318820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-21f52a7d-bed5-4a4b-b6dc-a007b088b27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:35.344912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f52a7d-bed5-4a4b-b6dc-a007b088b27c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:35.352569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dce35d-e2bf-4e63-a672-c11682e58471 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:35.353602 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dce35d-e2bf-4e63-a672-c11682e58471 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:35.354115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dce35d-e2bf-4e63-a672-c11682e58471 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:35.354582 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dce35d-e2bf-4e63-a672-c11682e58471 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:35.369522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dce35d-e2bf-4e63-a672-c11682e58471 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:35.372259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c338e58dc874c61b78364907397a85b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:35.372473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dce35d-e2bf-4e63-a672-c11682e58471 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:35.388402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c338e58dc874c61b78364907397a85b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:35.389521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c338e58dc874c61b78364907397a85b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:35.390654 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c338e58dc874c61b78364907397a85b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:35.391988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 867/1706] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:35.394954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dce35d-e2bf-4e63-a672-c11682e58471 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:35.414443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-16dce35d-e2bf-4e63-a672-c11682e58471 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:35.423674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c338e58dc874c61b78364907397a85b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:35.469485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 840/1707] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:08:34 2022] DELETE /identity/v3/users/4c338e58dc874c61b78364907397a85b => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:35.853305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dce35d-e2bf-4e63-a672-c11682e58471 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:36.041051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:36.299023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc8bf6-9e62-48cc-a03c-c9e7bf0d0fc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:36.300133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc8bf6-9e62-48cc-a03c-c9e7bf0d0fc5 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:36.301272 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc8bf6-9e62-48cc-a03c-c9e7bf0d0fc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:36.302081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc8bf6-9e62-48cc-a03c-c9e7bf0d0fc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:36.315235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cbdc8bf6-9e62-48cc-a03c-c9e7bf0d0fc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:36.659469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc8bf6-9e62-48cc-a03c-c9e7bf0d0fc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:36.700728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc8bf6-9e62-48cc-a03c-c9e7bf0d0fc5 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:36.714163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:36.715833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:36.790732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 868/1708] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 1397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:36.900708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06942004-90e8-4515-b4c8-ad2c8a268d6a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:36.901867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06942004-90e8-4515-b4c8-ad2c8a268d6a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:36.902634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06942004-90e8-4515-b4c8-ad2c8a268d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:37.062608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-06942004-90e8-4515-b4c8-ad2c8a268d6a None None] MFA Rules not processed for user `b37a4275e7c6441aa9e8bb0298f56589`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:37.072399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-06942004-90e8-4515-b4c8-ad2c8a268d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:37.117687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b8362-41db-4246-95be-e2f0bd99f2c0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:37.139499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:37.139499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:37.139499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:37.140002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] PATH_INFO: `/v3/projects/903bec1eca3c4c69b74e6f2961c31d9a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:37.148571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=903bec1eca3c4c69b74e6f2961c31d9a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:37.162102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:37.379419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 869/1709] 10.222.0.45 () {58 vars in 1015 bytes} [Fri Aug 19 21:08:36 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:37.466163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06942004-90e8-4515-b4c8-ad2c8a268d6a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:37.467706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-06942004-90e8-4515-b4c8-ad2c8a268d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:37.502760 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06942004-90e8-4515-b4c8-ad2c8a268d6a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:37.532925 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903bec1eca3c4c69b74e6f2961c31d9a', 'request_id': 'req-3c079571-87e2-4ffc-8726-4924d8e3a5c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:37.549048 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903bec1eca3c4c69b74e6f2961c31d9a', 'request_id': 'req-3c079571-87e2-4ffc-8726-4924d8e3a5c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:37.576307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903bec1eca3c4c69b74e6f2961c31d9a', 'request_id': 'req-3c079571-87e2-4ffc-8726-4924d8e3a5c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:37.601143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:37.637958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] The token cache is being invalidate because project 903bec1eca3c4c69b74e6f2961c31d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:08:37.641546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:37.648141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:37.665212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 841/1710] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:08:36 2022] DELETE /identity/v3/projects/903bec1eca3c4c69b74e6f2961c31d9a => generated 0 bytes in 951 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:37.668934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:37.678058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:37.849691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415e8c5-9efd-443e-8e8b-22ecef12c237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:37.852362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415e8c5-9efd-443e-8e8b-22ecef12c237 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:37.852727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415e8c5-9efd-443e-8e8b-22ecef12c237 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:37.855318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415e8c5-9efd-443e-8e8b-22ecef12c237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:37.864791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4415e8c5-9efd-443e-8e8b-22ecef12c237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:37.940841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415e8c5-9efd-443e-8e8b-22ecef12c237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:37.946034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415e8c5-9efd-443e-8e8b-22ecef12c237 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:37.984345 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 870/1711] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:08:38.023619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c079571-87e2-4ffc-8726-4924d8e3a5c8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:38.072463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:38.074650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:38.075155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:38.075645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] PATH_INFO: `/v3/users/03e50f525b6e4be5936cea7836225fea` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:38.083105 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=03e50f525b6e4be5936cea7836225fea)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:38.124680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:38.543436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03e50f525b6e4be5936cea7836225fea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:38.584196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03e50f525b6e4be5936cea7836225fea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:38.588104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03e50f525b6e4be5936cea7836225fea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:38.590257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03e50f525b6e4be5936cea7836225fea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:38.625278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03e50f525b6e4be5936cea7836225fea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:38.747733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 842/1712] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:08:37 2022] DELETE /identity/v3/users/03e50f525b6e4be5936cea7836225fea => generated 0 bytes in 1081 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:39.674692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74036291-bd6f-4cf1-b60e-6cb79272bd29 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:39.676214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74036291-bd6f-4cf1-b60e-6cb79272bd29 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:39.677095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74036291-bd6f-4cf1-b60e-6cb79272bd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:39.812966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-74036291-bd6f-4cf1-b60e-6cb79272bd29 None None] MFA Rules not processed for user `0809d96091c2476c83ec0c7c6c5a1509`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:39.820790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-74036291-bd6f-4cf1-b60e-6cb79272bd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:39.899033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:40.357009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 871/1713] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 685 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:40.380957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:40.386953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:40.866491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbf3e3-5ae2-45a2-8a26-af634f4f6e9f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:40.878363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93229f-e6a2-479c-9712-783801c4d345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:40.879026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93229f-e6a2-479c-9712-783801c4d345 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:40.880289 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93229f-e6a2-479c-9712-783801c4d345 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:40.880289 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93229f-e6a2-479c-9712-783801c4d345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:40.902496 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93229f-e6a2-479c-9712-783801c4d345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:40.904237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93229f-e6a2-479c-9712-783801c4d345 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:40.925497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 843/1714] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:42.284218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74036291-bd6f-4cf1-b60e-6cb79272bd29 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:42.286373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-74036291-bd6f-4cf1-b60e-6cb79272bd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:42.674911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74036291-bd6f-4cf1-b60e-6cb79272bd29 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:42.712946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:42.715462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:42.719243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:42.719783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] PATH_INFO: `/v3/projects/0f930cf47e1942f6b2d372a51f019926` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:42.726059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f930cf47e1942f6b2d372a51f019926)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:42.731650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:43.055137 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f930cf47e1942f6b2d372a51f019926', 'request_id': 'req-2f9f178f-b64c-4b99-957e-6e7048eec31c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:43.123648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f930cf47e1942f6b2d372a51f019926', 'request_id': 'req-2f9f178f-b64c-4b99-957e-6e7048eec31c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:43.188992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f930cf47e1942f6b2d372a51f019926', 'request_id': 'req-2f9f178f-b64c-4b99-957e-6e7048eec31c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:43.296128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] The token cache is being invalidate because project 0f930cf47e1942f6b2d372a51f019926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:08:43.296128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:43.300251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f9f178f-b64c-4b99-957e-6e7048eec31c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:43.309890 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 872/1715] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:08:42 2022] DELETE /identity/v3/projects/0f930cf47e1942f6b2d372a51f019926 => generated 0 bytes in 1027 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:43.316931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011e4ad-3196-4733-9cb4-be662e08e6d7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:43.317958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011e4ad-3196-4733-9cb4-be662e08e6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:43.317958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c011e4ad-3196-4733-9cb4-be662e08e6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:43.331253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:43.390031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c011e4ad-3196-4733-9cb4-be662e08e6d7 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:43.398358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c011e4ad-3196-4733-9cb4-be662e08e6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:43.632456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 844/1716] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:43.641139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a80ed1-ad60-44eb-9f23-d6cf0b67e8b4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:43.641600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a80ed1-ad60-44eb-9f23-d6cf0b67e8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:43.642087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a80ed1-ad60-44eb-9f23-d6cf0b67e8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:43.692389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-14a80ed1-ad60-44eb-9f23-d6cf0b67e8b4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:43.697597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-14a80ed1-ad60-44eb-9f23-d6cf0b67e8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:43.725722 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 873/1717] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:43.730043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011e4ad-3196-4733-9cb4-be662e08e6d7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:43.746640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c011e4ad-3196-4733-9cb4-be662e08e6d7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:43.765914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e620589-882d-4dfc-8ee0-1cdbe743d1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:43.766677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e620589-882d-4dfc-8ee0-1cdbe743d1e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:43.767040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e620589-882d-4dfc-8ee0-1cdbe743d1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:43.767388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e620589-882d-4dfc-8ee0-1cdbe743d1e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:43.768163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e620589-882d-4dfc-8ee0-1cdbe743d1e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:43.770020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e620589-882d-4dfc-8ee0-1cdbe743d1e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:43.775212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e620589-882d-4dfc-8ee0-1cdbe743d1e7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:43.821134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 845/1718] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:08:43.825479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a80ed1-ad60-44eb-9f23-d6cf0b67e8b4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:43.838186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a80ed1-ad60-44eb-9f23-d6cf0b67e8b4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:43.858866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af151bc0-17f2-4398-95da-710550513370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:43.859518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151bc0-17f2-4398-95da-710550513370 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:43.859799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151bc0-17f2-4398-95da-710550513370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:43.860076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af151bc0-17f2-4398-95da-710550513370 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:43.860736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af151bc0-17f2-4398-95da-710550513370 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:43.862878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af151bc0-17f2-4398-95da-710550513370 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:43.945410 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 874/1719] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:43 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:08:43.949948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e620589-882d-4dfc-8ee0-1cdbe743d1e7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:43.967748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e620589-882d-4dfc-8ee0-1cdbe743d1e7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:44.179177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef79a90-baaf-4e55-ab7d-ecee401a5228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:44.180294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef79a90-baaf-4e55-ab7d-ecee401a5228 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:44.180737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef79a90-baaf-4e55-ab7d-ecee401a5228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:44.181164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef79a90-baaf-4e55-ab7d-ecee401a5228 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:44.182072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef79a90-baaf-4e55-ab7d-ecee401a5228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:44.184329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef79a90-baaf-4e55-ab7d-ecee401a5228 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:44.245051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 846/1720] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:43 2022] POST /identity/v3/users => generated 282 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:44.246174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af151bc0-17f2-4398-95da-710550513370 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:44.262481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af151bc0-17f2-4398-95da-710550513370 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:44.283044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d7d0f-4a30-48b1-a769-68b93226ec7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:44.283911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d7d0f-4a30-48b1-a769-68b93226ec7a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:44.284370 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d7d0f-4a30-48b1-a769-68b93226ec7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:44.284793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d7d0f-4a30-48b1-a769-68b93226ec7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:44.285633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d7d0f-4a30-48b1-a769-68b93226ec7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:44.291023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d7d0f-4a30-48b1-a769-68b93226ec7a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:44.316794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:44.374019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 875/1721] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:08:44.378702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef79a90-baaf-4e55-ab7d-ecee401a5228 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:44.405418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef79a90-baaf-4e55-ab7d-ecee401a5228 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:44.416745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9f60f2-3c4d-44ed-88c1-6ecfeff3a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:44.418497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f60f2-3c4d-44ed-88c1-6ecfeff3a55a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:44.419649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f60f2-3c4d-44ed-88c1-6ecfeff3a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:44.420619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9f60f2-3c4d-44ed-88c1-6ecfeff3a55a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:44.422299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f60f2-3c4d-44ed-88c1-6ecfeff3a55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:44.434237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9f60f2-3c4d-44ed-88c1-6ecfeff3a55a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:44.518963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d7d0f-4a30-48b1-a769-68b93226ec7a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:44.519251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 847/1722] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:08:44.530546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d7d0f-4a30-48b1-a769-68b93226ec7a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:44.580139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35f68-febf-4614-af10-510fb0306cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:44.581092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35f68-febf-4614-af10-510fb0306cad admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:44.581531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35f68-febf-4614-af10-510fb0306cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:44.582002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd35f68-febf-4614-af10-510fb0306cad admin admin] PATH_INFO: `/v3/projects/9b295a8c0d934c778c94b1ed5f8b1a9d/users/7b42bba1d3294cec97f456841afa3ef8/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:44.639934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd35f68-febf-4614-af10-510fb0306cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b295a8c0d934c778c94b1ed5f8b1a9d, user_id=7b42bba1d3294cec97f456841afa3ef8, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:44.644448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd35f68-febf-4614-af10-510fb0306cad admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:44.693273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 876/1723] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:44 2022] PUT /identity/v3/projects/9b295a8c0d934c778c94b1ed5f8b1a9d/users/7b42bba1d3294cec97f456841afa3ef8/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:08:44.720675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed59927-34d5-4e55-abfd-94fad4555d91 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:44.721187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed59927-34d5-4e55-abfd-94fad4555d91 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:44.722245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed59927-34d5-4e55-abfd-94fad4555d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:44.839892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2ed59927-34d5-4e55-abfd-94fad4555d91 None None] MFA Rules not processed for user `7b42bba1d3294cec97f456841afa3ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:44.841568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2ed59927-34d5-4e55-abfd-94fad4555d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:45.065030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:45.090688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 848/1724] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:45.096518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35f68-febf-4614-af10-510fb0306cad admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:45.393831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd35f68-febf-4614-af10-510fb0306cad admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:45.419716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b15e0-52d3-4974-9d2d-6fce6f5325d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:45.421263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b15e0-52d3-4974-9d2d-6fce6f5325d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:45.422644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b15e0-52d3-4974-9d2d-6fce6f5325d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:45.423223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b15e0-52d3-4974-9d2d-6fce6f5325d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:45.424187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b15e0-52d3-4974-9d2d-6fce6f5325d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:45.426524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b15e0-52d3-4974-9d2d-6fce6f5325d7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:45.561645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 877/1725] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:45 2022] POST /identity/v3/projects => generated 365 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:45.565479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed59927-34d5-4e55-abfd-94fad4555d91 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:45.591649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed59927-34d5-4e55-abfd-94fad4555d91 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:45.784929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffddc89-b738-4e4f-96c0-834ff8f5ff7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:45.784929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffddc89-b738-4e4f-96c0-834ff8f5ff7f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:45.785329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffddc89-b738-4e4f-96c0-834ff8f5ff7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:45.785756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffddc89-b738-4e4f-96c0-834ff8f5ff7f admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:45.786514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffddc89-b738-4e4f-96c0-834ff8f5ff7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:45.789602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffddc89-b738-4e4f-96c0-834ff8f5ff7f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:45.817924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05b589-93b2-41bc-9779-a954f41b78ba None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:45.818425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05b589-93b2-41bc-9779-a954f41b78ba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:45.819924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05b589-93b2-41bc-9779-a954f41b78ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:45.866419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 849/1726] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:45 2022] POST /identity/v3/users => generated 289 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:45.867885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffddc89-b738-4e4f-96c0-834ff8f5ff7f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:45.888816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffddc89-b738-4e4f-96c0-834ff8f5ff7f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:45.920574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68012afd-ff16-436f-af26-51841d9f8be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:45.922251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68012afd-ff16-436f-af26-51841d9f8be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:45.922751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68012afd-ff16-436f-af26-51841d9f8be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:45.923734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68012afd-ff16-436f-af26-51841d9f8be1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:45.924630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68012afd-ff16-436f-af26-51841d9f8be1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:45.931005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ad05b589-93b2-41bc-9779-a954f41b78ba None None] MFA Rules not processed for user `8810b3e211d448cea9a506d9183dc13d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:45.934800 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ad05b589-93b2-41bc-9779-a954f41b78ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:45.936310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68012afd-ff16-436f-af26-51841d9f8be1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:45.969605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 850/1727] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:45.982900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68012afd-ff16-436f-af26-51841d9f8be1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:45.998744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68012afd-ff16-436f-af26-51841d9f8be1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:46.011544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3ff39-835b-4479-870a-722ef2706db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:46.012205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3ff39-835b-4479-870a-722ef2706db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:46.012598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3ff39-835b-4479-870a-722ef2706db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:46.013038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa3ff39-835b-4479-870a-722ef2706db4 admin admin] PATH_INFO: `/v3/projects/835ecb3974e14787bc16fa31a4e8a640/users/7a4a4e9106fd4525a5405d20a19cb559/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:46.059866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3ff39-835b-4479-870a-722ef2706db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=835ecb3974e14787bc16fa31a4e8a640, user_id=7a4a4e9106fd4525a5405d20a19cb559, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:46.063551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa3ff39-835b-4479-870a-722ef2706db4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:46.104768 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 851/1728] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:45 2022] PUT /identity/v3/projects/835ecb3974e14787bc16fa31a4e8a640/users/7a4a4e9106fd4525a5405d20a19cb559/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:46.113968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3ff39-835b-4479-870a-722ef2706db4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:46.219723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 878/1729] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:46.232669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5091d3-a282-4515-ac27-bf7641ccd0fd None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:46.232958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5091d3-a282-4515-ac27-bf7641ccd0fd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:46.233237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5091d3-a282-4515-ac27-bf7641ccd0fd None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:46.236268 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 879/1730] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:08:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:08:46.255610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5091d3-a282-4515-ac27-bf7641ccd0fd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:46.264601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fc5091d3-a282-4515-ac27-bf7641ccd0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:46.482742 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:46.565588 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5091d3-a282-4515-ac27-bf7641ccd0fd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:46.594038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa3ff39-835b-4479-870a-722ef2706db4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:46.596079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be676185-e4b0-4497-a1e2-9836fc999ac5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:46.597052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be676185-e4b0-4497-a1e2-9836fc999ac5 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:46.597540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be676185-e4b0-4497-a1e2-9836fc999ac5 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:46.598073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be676185-e4b0-4497-a1e2-9836fc999ac5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:46.628710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6d79a6-6114-4de4-aca9-178e1688fee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:46.629823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d79a6-6114-4de4-aca9-178e1688fee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:46.630576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d79a6-6114-4de4-aca9-178e1688fee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:46.632978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6d79a6-6114-4de4-aca9-178e1688fee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:46.634471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d79a6-6114-4de4-aca9-178e1688fee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:46.644560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6d79a6-6114-4de4-aca9-178e1688fee8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:46.710389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 852/1731] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 599 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:46.713334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2bdbf-d3a4-4a8d-a1ff-8b3ca6c8fa0d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:46.718860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2bdbf-d3a4-4a8d-a1ff-8b3ca6c8fa0d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:46.719331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2bdbf-d3a4-4a8d-a1ff-8b3ca6c8fa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:46.803761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be676185-e4b0-4497-a1e2-9836fc999ac5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:46.812001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be676185-e4b0-4497-a1e2-9836fc999ac5 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:46.863895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-79c2bdbf-d3a4-4a8d-a1ff-8b3ca6c8fa0d None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:46.878360 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 880/1732] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 623 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:08:46.880503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be676185-e4b0-4497-a1e2-9836fc999ac5 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:46.883196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-79c2bdbf-d3a4-4a8d-a1ff-8b3ca6c8fa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:46.925542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be676185-e4b0-4497-a1e2-9836fc999ac5 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:46.951833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 853/1733] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:08:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:46.952293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fc738-4f63-49ff-9dfc-8cde67f7b4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:46.953242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fc738-4f63-49ff-9dfc-8cde67f7b4db admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:46.957043 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fc738-4f63-49ff-9dfc-8cde67f7b4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:46.962028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231fc738-4f63-49ff-9dfc-8cde67f7b4db admin admin] PATH_INFO: `/v3/projects/835ecb3974e14787bc16fa31a4e8a640/users/7a4a4e9106fd4525a5405d20a19cb559/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:46.970196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fc738-4f63-49ff-9dfc-8cde67f7b4db admin admin] RBAC: Authorizing `identity:create_grant(project_id=835ecb3974e14787bc16fa31a4e8a640, user_id=7a4a4e9106fd4525a5405d20a19cb559, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:46.974262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231fc738-4f63-49ff-9dfc-8cde67f7b4db admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:47.002591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 881/1734] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:46 2022] PUT /identity/v3/projects/835ecb3974e14787bc16fa31a4e8a640/users/7a4a4e9106fd4525a5405d20a19cb559/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:47.019911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2cb6f-a99f-400b-8745-0584851d4abb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:47.020228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2cb6f-a99f-400b-8745-0584851d4abb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:47.020606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca2cb6f-a99f-400b-8745-0584851d4abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:47.156020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fca2cb6f-a99f-400b-8745-0584851d4abb None None] MFA Rules not processed for user `7a4a4e9106fd4525a5405d20a19cb559`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:47.169169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fca2cb6f-a99f-400b-8745-0584851d4abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:47.360806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:47.549093 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 854/1735] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 530 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:47.567394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fc738-4f63-49ff-9dfc-8cde67f7b4db admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:47.570673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-231fc738-4f63-49ff-9dfc-8cde67f7b4db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:47.747135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231fc738-4f63-49ff-9dfc-8cde67f7b4db admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:47.762239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8106d-8156-4d05-b81b-f3bc61d29f11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:47.763016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8106d-8156-4d05-b81b-f3bc61d29f11 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:47.763413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8106d-8156-4d05-b81b-f3bc61d29f11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:47.763792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b8106d-8156-4d05-b81b-f3bc61d29f11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:47.787868 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8106d-8156-4d05-b81b-f3bc61d29f11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:47.797624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b8106d-8156-4d05-b81b-f3bc61d29f11 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:47.831780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 882/1736] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:47.976440 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2cb6f-a99f-400b-8745-0584851d4abb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:48.003897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca2cb6f-a99f-400b-8745-0584851d4abb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:48.015589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579fe99-dbdb-4f10-af34-86fad6da436f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:48.016577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579fe99-dbdb-4f10-af34-86fad6da436f service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:48.017048 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579fe99-dbdb-4f10-af34-86fad6da436f service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:48.017484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579fe99-dbdb-4f10-af34-86fad6da436f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:48.214929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579fe99-dbdb-4f10-af34-86fad6da436f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:48.216486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579fe99-dbdb-4f10-af34-86fad6da436f service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:48.236040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 855/1737] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:08:49.465291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8106d-8156-4d05-b81b-f3bc61d29f11 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:49.466866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-65b8106d-8156-4d05-b81b-f3bc61d29f11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:49.817910 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b8106d-8156-4d05-b81b-f3bc61d29f11 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:49.842894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fab7c-3872-4e59-ae46-41247b523698 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:49.844087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fab7c-3872-4e59-ae46-41247b523698 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:49.844529 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fab7c-3872-4e59-ae46-41247b523698 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:49.844949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fab7c-3872-4e59-ae46-41247b523698 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:49.883415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fab7c-3872-4e59-ae46-41247b523698 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:49.885547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fab7c-3872-4e59-ae46-41247b523698 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:49.927708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 883/1738] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:08:52.976742 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378280ef-d186-4ede-93be-be67e9a6970e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:52.978043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378280ef-d186-4ede-93be-be67e9a6970e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:52.979612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378280ef-d186-4ede-93be-be67e9a6970e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:52.983245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 856/1739] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:08:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:08:53.034043 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb0c04-7511-4a47-8bc8-0e89e5f6da95 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:53.035004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb0c04-7511-4a47-8bc8-0e89e5f6da95 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:53.035629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb0c04-7511-4a47-8bc8-0e89e5f6da95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:53.055491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:53.079480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378280ef-d186-4ede-93be-be67e9a6970e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:53.083702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-378280ef-d186-4ede-93be-be67e9a6970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:53.107868 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-54bb0c04-7511-4a47-8bc8-0e89e5f6da95 None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:53.120775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-54bb0c04-7511-4a47-8bc8-0e89e5f6da95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:53.153939 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 884/1740] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:08:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:53.241969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378280ef-d186-4ede-93be-be67e9a6970e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:53.252646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:53.255713 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:53.257062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:53.259230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] PATH_INFO: `/v3/users/750b61462478471fa4e1e2f45c269109` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:53.262616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=750b61462478471fa4e1e2f45c269109)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:53.271162 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:53.396933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '750b61462478471fa4e1e2f45c269109'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:53.414484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '750b61462478471fa4e1e2f45c269109'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:53.416383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '750b61462478471fa4e1e2f45c269109'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:53.418146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '750b61462478471fa4e1e2f45c269109'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:53.447635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bd0e102-7e89-44ab-b3a5-ca2e7a030591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '750b61462478471fa4e1e2f45c269109'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:53.455339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:53.486399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 857/1741] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:08:53 2022] DELETE /identity/v3/users/750b61462478471fa4e1e2f45c269109 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:53.492423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551bd6b-45db-4368-ada0-2eb4afc9a15a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:53.493021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551bd6b-45db-4368-ada0-2eb4afc9a15a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:53.493643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551bd6b-45db-4368-ada0-2eb4afc9a15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:53.548571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0551bd6b-45db-4368-ada0-2eb4afc9a15a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:53.563053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0551bd6b-45db-4368-ada0-2eb4afc9a15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:53.825703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 885/1742] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:54.138433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d7f80-9186-4fd9-ba0e-894aadff7e8c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:54.138433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d7f80-9186-4fd9-ba0e-894aadff7e8c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:54.138433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d7f80-9186-4fd9-ba0e-894aadff7e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:54.183193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-365d7f80-9186-4fd9-ba0e-894aadff7e8c None None] MFA Rules not processed for user `7f46019e78cc4e89bf877a4f5a7a8fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:54.184574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-365d7f80-9186-4fd9-ba0e-894aadff7e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:54.348667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 858/1743] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:54.414036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551bd6b-45db-4368-ada0-2eb4afc9a15a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:54.415541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0551bd6b-45db-4368-ada0-2eb4afc9a15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:54.523318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0551bd6b-45db-4368-ada0-2eb4afc9a15a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:54.531878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11306ae9-faf2-46d7-8959-61b627444210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:54.534673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11306ae9-faf2-46d7-8959-61b627444210 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:54.535481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11306ae9-faf2-46d7-8959-61b627444210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:54.536205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11306ae9-faf2-46d7-8959-61b627444210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:54.556092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11306ae9-faf2-46d7-8959-61b627444210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:54.557637 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11306ae9-faf2-46d7-8959-61b627444210 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:54.572687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 886/1744] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:08:54.908765 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d7f80-9186-4fd9-ba0e-894aadff7e8c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:54.911973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-365d7f80-9186-4fd9-ba0e-894aadff7e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:55.148863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d7f80-9186-4fd9-ba0e-894aadff7e8c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:55.178940 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a8d22-be4b-41fe-84dc-7735c3e1b06e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:55.182255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a8d22-be4b-41fe-84dc-7735c3e1b06e service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:55.183040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a8d22-be4b-41fe-84dc-7735c3e1b06e service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:55.183750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a8d22-be4b-41fe-84dc-7735c3e1b06e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:55.187459 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe9a8d22-be4b-41fe-84dc-7735c3e1b06e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:55.221850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a8d22-be4b-41fe-84dc-7735c3e1b06e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:55.227020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a8d22-be4b-41fe-84dc-7735c3e1b06e service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:55.239267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 859/1745] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:08:55.816925 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11306ae9-faf2-46d7-8959-61b627444210 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:55.819820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11306ae9-faf2-46d7-8959-61b627444210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:55.857548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11306ae9-faf2-46d7-8959-61b627444210 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:55.865245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:55.866250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:55.866990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:55.867413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] PATH_INFO: `/v3/projects/6194aa367f994e3e8f90baa7e5d3c97b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:55.870261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6194aa367f994e3e8f90baa7e5d3c97b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:55.873371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:55.961717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6194aa367f994e3e8f90baa7e5d3c97b', 'request_id': 'req-bed58593-e283-4696-9968-4e20bd264189'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:55.979586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6194aa367f994e3e8f90baa7e5d3c97b', 'request_id': 'req-bed58593-e283-4696-9968-4e20bd264189'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:55.988346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6194aa367f994e3e8f90baa7e5d3c97b', 'request_id': 'req-bed58593-e283-4696-9968-4e20bd264189'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:56.005710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] The token cache is being invalidate because project 6194aa367f994e3e8f90baa7e5d3c97b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:08:56.006301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:56.007372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bed58593-e283-4696-9968-4e20bd264189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:56.009417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 887/1746] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:08:55 2022] DELETE /identity/v3/projects/6194aa367f994e3e8f90baa7e5d3c97b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:56.014364 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a8d22-be4b-41fe-84dc-7735c3e1b06e service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:56.026787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a8d22-be4b-41fe-84dc-7735c3e1b06e service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:56.115047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:56.116044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:56.116322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:56.116587 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] PATH_INFO: `/v3/users/97099ad0c973491a8fe24f2167ec4b08` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:56.117960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97099ad0c973491a8fe24f2167ec4b08)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:56.120000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:56.142429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:56.196571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97099ad0c973491a8fe24f2167ec4b08'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:56.213422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97099ad0c973491a8fe24f2167ec4b08'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:56.214524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97099ad0c973491a8fe24f2167ec4b08'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:56.215328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97099ad0c973491a8fe24f2167ec4b08'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:56.228629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97099ad0c973491a8fe24f2167ec4b08'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:56.237491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c513169-6cf7-4459-be72-b310c893e0cf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:56.238026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c513169-6cf7-4459-be72-b310c893e0cf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:56.238516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c513169-6cf7-4459-be72-b310c893e0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:56.258228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 860/1747] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:08:56 2022] DELETE /identity/v3/users/97099ad0c973491a8fe24f2167ec4b08 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:56.278450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5c513169-6cf7-4459-be72-b310c893e0cf None None] MFA Rules not processed for user `0809d96091c2476c83ec0c7c6c5a1509`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:56.279596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5c513169-6cf7-4459-be72-b310c893e0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:56.306364 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:56.386042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 888/1748] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:57.328614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:57.356436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:57.520011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d925b2-1f18-4bdb-a47d-8100d2c6a998 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:57.527592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:57.528722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:57.529195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:57.529511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] PATH_INFO: `/v3/projects/8bdf0a4289c846ca9f5c87bc334fd9d6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:57.531494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bdf0a4289c846ca9f5c87bc334fd9d6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:57.534739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:57.619208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bdf0a4289c846ca9f5c87bc334fd9d6', 'request_id': 'req-472be777-f9a6-45b9-8393-ad70c7d3cc81'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:57.627464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bdf0a4289c846ca9f5c87bc334fd9d6', 'request_id': 'req-472be777-f9a6-45b9-8393-ad70c7d3cc81'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:57.637666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bdf0a4289c846ca9f5c87bc334fd9d6', 'request_id': 'req-472be777-f9a6-45b9-8393-ad70c7d3cc81'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:57.671780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] The token cache is being invalidate because project 8bdf0a4289c846ca9f5c87bc334fd9d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:08:57.672095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:57.672887 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-472be777-f9a6-45b9-8393-ad70c7d3cc81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:08:57.674999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 861/1749] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:08:57 2022] DELETE /identity/v3/projects/8bdf0a4289c846ca9f5c87bc334fd9d6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:57.684557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed737559-008e-4607-bd22-556b68baf045 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:57.685102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed737559-008e-4607-bd22-556b68baf045 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:57.685532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed737559-008e-4607-bd22-556b68baf045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:57.742498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ed737559-008e-4607-bd22-556b68baf045 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:57.745020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ed737559-008e-4607-bd22-556b68baf045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:57.872515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 889/1750] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:57.887309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18b2d9-a030-4a40-b7b1-02ca90b044e5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:57.888119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18b2d9-a030-4a40-b7b1-02ca90b044e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:57.889109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18b2d9-a030-4a40-b7b1-02ca90b044e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:57.921115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3d18b2d9-a030-4a40-b7b1-02ca90b044e5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:57.924080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3d18b2d9-a030-4a40-b7b1-02ca90b044e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:57.938961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 862/1751] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:57.944997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed737559-008e-4607-bd22-556b68baf045 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:57.959677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed737559-008e-4607-bd22-556b68baf045 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:57.967505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6e8ab8-00b3-4f73-a1fb-e9a349557a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:57.968444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e8ab8-00b3-4f73-a1fb-e9a349557a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:57.968869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e8ab8-00b3-4f73-a1fb-e9a349557a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:57.969308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6e8ab8-00b3-4f73-a1fb-e9a349557a23 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:57.970181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e8ab8-00b3-4f73-a1fb-e9a349557a23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:08:57.970699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e8ab8-00b3-4f73-a1fb-e9a349557a23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:57.972856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6e8ab8-00b3-4f73-a1fb-e9a349557a23 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:57.987735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 890/1752] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:08:57.995178 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18b2d9-a030-4a40-b7b1-02ca90b044e5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:58.006891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18b2d9-a030-4a40-b7b1-02ca90b044e5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:58.012946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e938635-d828-457e-822f-6f43b4d915ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:58.013545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e938635-d828-457e-822f-6f43b4d915ca admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:58.014022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e938635-d828-457e-822f-6f43b4d915ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:58.014487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e938635-d828-457e-822f-6f43b4d915ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:58.015195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e938635-d828-457e-822f-6f43b4d915ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:58.017455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e938635-d828-457e-822f-6f43b4d915ca admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:58.025462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee796d09-5912-48f1-a909-6e520b459b8c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:58.025721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee796d09-5912-48f1-a909-6e520b459b8c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:58.026064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee796d09-5912-48f1-a909-6e520b459b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:58.044345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:58.051634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ee796d09-5912-48f1-a909-6e520b459b8c None None] MFA Rules not processed for user `8810b3e211d448cea9a506d9183dc13d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:58.053426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ee796d09-5912-48f1-a909-6e520b459b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:58.067717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 863/1753] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:57 2022] POST /identity/v3/projects => generated 365 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:58.070593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e938635-d828-457e-822f-6f43b4d915ca admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:58.151423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 891/1754] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:58.186256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e938635-d828-457e-822f-6f43b4d915ca admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:58.192227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7bfba0-c1eb-42b4-955f-d39fb92db123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:58.193161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bfba0-c1eb-42b4-955f-d39fb92db123 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:58.193745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bfba0-c1eb-42b4-955f-d39fb92db123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:58.194162 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7bfba0-c1eb-42b4-955f-d39fb92db123 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:58.194893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7bfba0-c1eb-42b4-955f-d39fb92db123 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:58.197520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7bfba0-c1eb-42b4-955f-d39fb92db123 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:58.241924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 864/1755] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:58 2022] POST /identity/v3/users => generated 283 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:58.245105 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee796d09-5912-48f1-a909-6e520b459b8c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:58.268548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee796d09-5912-48f1-a909-6e520b459b8c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:58.276098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1843fd2-af7f-4772-96af-8288e4e9c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:58.276869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1843fd2-af7f-4772-96af-8288e4e9c97c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:58.278306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1843fd2-af7f-4772-96af-8288e4e9c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:58.278306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1843fd2-af7f-4772-96af-8288e4e9c97c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:58.278869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1843fd2-af7f-4772-96af-8288e4e9c97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:58.280901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1843fd2-af7f-4772-96af-8288e4e9c97c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:58.295310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 892/1756] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:58.299672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7bfba0-c1eb-42b4-955f-d39fb92db123 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:58.310973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7bfba0-c1eb-42b4-955f-d39fb92db123 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:58.316506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43525e84-01a9-4bc1-abcd-0c088194c8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:58.317227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43525e84-01a9-4bc1-abcd-0c088194c8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:58.317506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43525e84-01a9-4bc1-abcd-0c088194c8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:58.317865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43525e84-01a9-4bc1-abcd-0c088194c8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:58.319120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43525e84-01a9-4bc1-abcd-0c088194c8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:58.320791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43525e84-01a9-4bc1-abcd-0c088194c8a6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:58.333371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 865/1757] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:58.338755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1843fd2-af7f-4772-96af-8288e4e9c97c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:58.352473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1843fd2-af7f-4772-96af-8288e4e9c97c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:58.357498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9698704-2dd6-43f2-9162-993e35277d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:58.358245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9698704-2dd6-43f2-9162-993e35277d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:58.358519 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9698704-2dd6-43f2-9162-993e35277d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:58.358792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9698704-2dd6-43f2-9162-993e35277d8f admin admin] PATH_INFO: `/v3/projects/5a8c334e33dc4b40a9c47ab647277f62/users/434df43b432d426cbc362de1ef6741f5/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:58.377463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9698704-2dd6-43f2-9162-993e35277d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8c334e33dc4b40a9c47ab647277f62, user_id=434df43b432d426cbc362de1ef6741f5, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:58.380536 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9698704-2dd6-43f2-9162-993e35277d8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:58.401568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 893/1758] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:58 2022] PUT /identity/v3/projects/5a8c334e33dc4b40a9c47ab647277f62/users/434df43b432d426cbc362de1ef6741f5/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:58.414671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226f27-5434-42bc-b49c-de2a028948c3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:58.415098 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226f27-5434-42bc-b49c-de2a028948c3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:58.415467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b226f27-5434-42bc-b49c-de2a028948c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:58.450996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2b226f27-5434-42bc-b49c-de2a028948c3 None None] MFA Rules not processed for user `434df43b432d426cbc362de1ef6741f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:58.452723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b226f27-5434-42bc-b49c-de2a028948c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:58.548402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 866/1759] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:08:58.553896 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9698704-2dd6-43f2-9162-993e35277d8f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:58.590594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:58.641404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9698704-2dd6-43f2-9162-993e35277d8f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:58.649201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46ef1f-0270-4930-a0e2-00164e7dc87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:58.649980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46ef1f-0270-4930-a0e2-00164e7dc87b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:58.650379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46ef1f-0270-4930-a0e2-00164e7dc87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:58.650796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f46ef1f-0270-4930-a0e2-00164e7dc87b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:58.651557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46ef1f-0270-4930-a0e2-00164e7dc87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:58.654059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f46ef1f-0270-4930-a0e2-00164e7dc87b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:58.704261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 894/1760] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:08:58 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:58.710556 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b226f27-5434-42bc-b49c-de2a028948c3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:58.726913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b226f27-5434-42bc-b49c-de2a028948c3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:58.851658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f924b7f-0688-4aa1-8220-fe5fffeb8766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:58.852666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f924b7f-0688-4aa1-8220-fe5fffeb8766 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:58.853105 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f924b7f-0688-4aa1-8220-fe5fffeb8766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:58.853553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f924b7f-0688-4aa1-8220-fe5fffeb8766 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:58.854733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f924b7f-0688-4aa1-8220-fe5fffeb8766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:58.862944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f924b7f-0688-4aa1-8220-fe5fffeb8766 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:58.893988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:08:58.937254 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 867/1761] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:08:58 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:08:58.941727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46ef1f-0270-4930-a0e2-00164e7dc87b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:58.980483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f46ef1f-0270-4930-a0e2-00164e7dc87b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:59.003466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cbe89-aa1c-4eaa-befd-1e0cd78dc6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:59.004633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cbe89-aa1c-4eaa-befd-1e0cd78dc6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:59.008233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cbe89-aa1c-4eaa-befd-1e0cd78dc6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:59.008833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cbe89-aa1c-4eaa-befd-1e0cd78dc6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:59.010072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cbe89-aa1c-4eaa-befd-1e0cd78dc6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:59.020148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cbe89-aa1c-4eaa-befd-1e0cd78dc6fc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:59.046102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 895/1762] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:08:59.051207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f924b7f-0688-4aa1-8220-fe5fffeb8766 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:59.091168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f924b7f-0688-4aa1-8220-fe5fffeb8766 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:59.096638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77947080-4877-4773-b3ef-f936cf15cee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:59.097662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77947080-4877-4773-b3ef-f936cf15cee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:59.098163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77947080-4877-4773-b3ef-f936cf15cee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:59.098618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77947080-4877-4773-b3ef-f936cf15cee8 admin admin] PATH_INFO: `/v3/projects/7d943e21a0e6470cbf8812c0cb4f25b3/users/a9bdecde34454140ae2ded3e79575a6b/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:59.139046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77947080-4877-4773-b3ef-f936cf15cee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d943e21a0e6470cbf8812c0cb4f25b3, user_id=a9bdecde34454140ae2ded3e79575a6b, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:59.141916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77947080-4877-4773-b3ef-f936cf15cee8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:59.177879 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 868/1763] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:59 2022] PUT /identity/v3/projects/7d943e21a0e6470cbf8812c0cb4f25b3/users/a9bdecde34454140ae2ded3e79575a6b/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:59.180040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cbe89-aa1c-4eaa-befd-1e0cd78dc6fc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:59.231715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cbe89-aa1c-4eaa-befd-1e0cd78dc6fc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:59.483878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13e9bf-15f0-4eef-a809-88ade79ec601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:59.485163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13e9bf-15f0-4eef-a809-88ade79ec601 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:59.485664 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13e9bf-15f0-4eef-a809-88ade79ec601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:59.486730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13e9bf-15f0-4eef-a809-88ade79ec601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:59.490885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13e9bf-15f0-4eef-a809-88ade79ec601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:59.503912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13e9bf-15f0-4eef-a809-88ade79ec601 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:59.578964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 896/1764] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:08:59.584798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77947080-4877-4773-b3ef-f936cf15cee8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:59.610639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77947080-4877-4773-b3ef-f936cf15cee8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:08:59.637359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ef750-6f32-4774-a80f-9c5f099ffa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:08:59.647757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ef750-6f32-4774-a80f-9c5f099ffa4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:59.648426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ef750-6f32-4774-a80f-9c5f099ffa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:59.648775 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ef750-6f32-4774-a80f-9c5f099ffa4e admin admin] PATH_INFO: `/v3/projects/7d943e21a0e6470cbf8812c0cb4f25b3/users/a9bdecde34454140ae2ded3e79575a6b/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:59.655182 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ef750-6f32-4774-a80f-9c5f099ffa4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d943e21a0e6470cbf8812c0cb4f25b3, user_id=a9bdecde34454140ae2ded3e79575a6b, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:08:59.669676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ef750-6f32-4774-a80f-9c5f099ffa4e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:08:59.699617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 869/1765] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:08:59 2022] PUT /identity/v3/projects/7d943e21a0e6470cbf8812c0cb4f25b3/users/a9bdecde34454140ae2ded3e79575a6b/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:08:59.720755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d0898-8c25-449d-95bd-2098fc24c32d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:08:59.721584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d0898-8c25-449d-95bd-2098fc24c32d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:08:59.722463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d0898-8c25-449d-95bd-2098fc24c32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:08:59.775310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cd2d0898-8c25-449d-95bd-2098fc24c32d None None] MFA Rules not processed for user `a9bdecde34454140ae2ded3e79575a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:08:59.780590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cd2d0898-8c25-449d-95bd-2098fc24c32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:08:59.893317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 897/1766] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:08:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:08:59.906382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ef750-6f32-4774-a80f-9c5f099ffa4e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:08:59.908836 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-847ef750-6f32-4774-a80f-9c5f099ffa4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:00.030546 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ef750-6f32-4774-a80f-9c5f099ffa4e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:00.043643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c251c5a3-f57b-4f34-a6b4-3a20e8a5c36b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:00.044551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251c5a3-f57b-4f34-a6b4-3a20e8a5c36b service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:00.045002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251c5a3-f57b-4f34-a6b4-3a20e8a5c36b service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:00.045526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c251c5a3-f57b-4f34-a6b4-3a20e8a5c36b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:00.067233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251c5a3-f57b-4f34-a6b4-3a20e8a5c36b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:00.069727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c251c5a3-f57b-4f34-a6b4-3a20e8a5c36b service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:00.089215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 870/1767] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:08:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:09:01.224128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d0898-8c25-449d-95bd-2098fc24c32d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:01.274945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:01.402829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d0898-8c25-449d-95bd-2098fc24c32d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:01.410724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:01.411971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:01.412651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:01.413822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] PATH_INFO: `/v3/users/434df43b432d426cbc362de1ef6741f5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:01.415628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] RBAC: Authorizing `identity:delete_user(user_id=434df43b432d426cbc362de1ef6741f5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:01.417490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 19 21:09:01.417490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Aug 19 21:09:01.417490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: Aug 19 21:09:01.419889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:01.597560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434df43b432d426cbc362de1ef6741f5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:01.606695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434df43b432d426cbc362de1ef6741f5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:01.608321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434df43b432d426cbc362de1ef6741f5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:01.625065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434df43b432d426cbc362de1ef6741f5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:01.667032 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434df43b432d426cbc362de1ef6741f5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:01.691401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 898/1768] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:01 2022] DELETE /identity/v3/users/434df43b432d426cbc362de1ef6741f5 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:01.696933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be80c-58ed-4c74-b9f3-9a71b3514e90 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:01.697450 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be80c-58ed-4c74-b9f3-9a71b3514e90 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:01.697536 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be80c-58ed-4c74-b9f3-9a71b3514e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:01.719241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-872be80c-58ed-4c74-b9f3-9a71b3514e90 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:01.720630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-872be80c-58ed-4c74-b9f3-9a71b3514e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:01.793731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 871/1769] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:01.805160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:01.806708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:01.902045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecdd41-5fc5-4aa0-8033-fedb0f4e533d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:01.906526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8460c1-9c25-41a8-a31e-323d927e5e78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:01.907261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8460c1-9c25-41a8-a31e-323d927e5e78 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:01.907837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8460c1-9c25-41a8-a31e-323d927e5e78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:01.908203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8460c1-9c25-41a8-a31e-323d927e5e78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:01.923994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8460c1-9c25-41a8-a31e-323d927e5e78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:01.925808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8460c1-9c25-41a8-a31e-323d927e5e78 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:01.939007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 899/1770] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:02.413177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be80c-58ed-4c74-b9f3-9a71b3514e90 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:02.415132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-872be80c-58ed-4c74-b9f3-9a71b3514e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:02.433972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be80c-58ed-4c74-b9f3-9a71b3514e90 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:02.440078 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:02.441016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:02.441426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:02.441935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] PATH_INFO: `/v3/projects/5a8c334e33dc4b40a9c47ab647277f62` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:02.444445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8c334e33dc4b40a9c47ab647277f62)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:02.447471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:02.516152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8c334e33dc4b40a9c47ab647277f62', 'request_id': 'req-462c9642-18e8-49b8-b5f8-787e10ad5de8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:02.528340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8c334e33dc4b40a9c47ab647277f62', 'request_id': 'req-462c9642-18e8-49b8-b5f8-787e10ad5de8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:02.530938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=3,keystone:SELECT=93,keystone:DELETE=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:02.539927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8c334e33dc4b40a9c47ab647277f62', 'request_id': 'req-462c9642-18e8-49b8-b5f8-787e10ad5de8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:02.571455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] The token cache is being invalidate because project 5a8c334e33dc4b40a9c47ab647277f62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:02.571806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:02.579840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:02.587421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 872/1771] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:02 2022] DELETE /identity/v3/projects/5a8c334e33dc4b40a9c47ab647277f62 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:02.589672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8460c1-9c25-41a8-a31e-323d927e5e78 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:02.607412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8460c1-9c25-41a8-a31e-323d927e5e78 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:02.680530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:02.742196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:02.743913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:02.744743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:02.745283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] PATH_INFO: `/v3/users/a9bdecde34454140ae2ded3e79575a6b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:02.747614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9bdecde34454140ae2ded3e79575a6b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:02.759383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:02.967058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9bdecde34454140ae2ded3e79575a6b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:02.969125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9bdecde34454140ae2ded3e79575a6b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:02.979607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9bdecde34454140ae2ded3e79575a6b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:03.007417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9bdecde34454140ae2ded3e79575a6b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:03.048712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2322ccb-afe3-4f79-8a7b-2c9737733f9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9bdecde34454140ae2ded3e79575a6b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:03.067471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:03.070117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:03.088594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 900/1772] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:02 2022] DELETE /identity/v3/users/a9bdecde34454140ae2ded3e79575a6b => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:03.116611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c9642-18e8-49b8-b5f8-787e10ad5de8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:03.317419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:03.321936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:03.325348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:03.326155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] PATH_INFO: `/v3/users/7b42bba1d3294cec97f456841afa3ef8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:03.330370 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b42bba1d3294cec97f456841afa3ef8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:03.336236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:03.576339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d797415-b1a5-45dc-8808-072f8be94348 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:03.576856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d797415-b1a5-45dc-8808-072f8be94348 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:03.577699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d797415-b1a5-45dc-8808-072f8be94348 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:03.582820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 901/1773] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:09:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:09:03.609757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a5bf7-4610-40ea-9aa3-ad1aa5e201df None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:03.610470 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a5bf7-4610-40ea-9aa3-ad1aa5e201df None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:03.610470 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a5bf7-4610-40ea-9aa3-ad1aa5e201df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:03.679477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f87a5bf7-4610-40ea-9aa3-ad1aa5e201df None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:03.681051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f87a5bf7-4610-40ea-9aa3-ad1aa5e201df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:03.681271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b42bba1d3294cec97f456841afa3ef8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:03.714039 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b42bba1d3294cec97f456841afa3ef8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:03.715301 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b42bba1d3294cec97f456841afa3ef8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:03.717073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b42bba1d3294cec97f456841afa3ef8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:03.762225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-349286d5-9aed-4039-be3f-617eaa0e8a6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b42bba1d3294cec97f456841afa3ef8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:03.859791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 873/1774] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:03 2022] DELETE /identity/v3/users/7b42bba1d3294cec97f456841afa3ef8 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:03.869030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968511f7-7ccc-4832-9f37-c7780a485e47 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:03.869408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968511f7-7ccc-4832-9f37-c7780a485e47 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:03.876351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968511f7-7ccc-4832-9f37-c7780a485e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:03.929734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-968511f7-7ccc-4832-9f37-c7780a485e47 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:03.933476 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-968511f7-7ccc-4832-9f37-c7780a485e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:03.957168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 902/1775] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:03.977235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a5bf7-4610-40ea-9aa3-ad1aa5e201df None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:03.979093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f87a5bf7-4610-40ea-9aa3-ad1aa5e201df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:04.013566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a5bf7-4610-40ea-9aa3-ad1aa5e201df None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:04.114166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 874/1776] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:04.138969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:04.157413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968511f7-7ccc-4832-9f37-c7780a485e47 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:04.160765 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-968511f7-7ccc-4832-9f37-c7780a485e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:04.175755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:04.184414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80caf5-4c3a-4e5a-9d32-91bc9267c576 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:04.185514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80caf5-4c3a-4e5a-9d32-91bc9267c576 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:04.186048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80caf5-4c3a-4e5a-9d32-91bc9267c576 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:04.186573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80caf5-4c3a-4e5a-9d32-91bc9267c576 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:04.211482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80caf5-4c3a-4e5a-9d32-91bc9267c576 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:04.213407 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da80caf5-4c3a-4e5a-9d32-91bc9267c576 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:04.352111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 903/1777] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:04.505317 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968511f7-7ccc-4832-9f37-c7780a485e47 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:04.529568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7752ba-0869-4e19-b0b4-ca163957256a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:04.530405 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7752ba-0869-4e19-b0b4-ca163957256a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:04.530634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7752ba-0869-4e19-b0b4-ca163957256a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:04.530914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7752ba-0869-4e19-b0b4-ca163957256a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:04.553454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7752ba-0869-4e19-b0b4-ca163957256a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:04.557024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7752ba-0869-4e19-b0b4-ca163957256a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:04.588872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 875/1778] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:05.251944 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80caf5-4c3a-4e5a-9d32-91bc9267c576 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:05.254856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-da80caf5-4c3a-4e5a-9d32-91bc9267c576 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:05.297625 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80caf5-4c3a-4e5a-9d32-91bc9267c576 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:05.308299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:05.310601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:05.312239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:05.313175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] PATH_INFO: `/v3/projects/7d943e21a0e6470cbf8812c0cb4f25b3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:05.316045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d943e21a0e6470cbf8812c0cb4f25b3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:05.323503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:05.501362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d943e21a0e6470cbf8812c0cb4f25b3', 'request_id': 'req-b5c6767d-afb5-4514-af3a-37d07a52da8b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:05.514875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d943e21a0e6470cbf8812c0cb4f25b3', 'request_id': 'req-b5c6767d-afb5-4514-af3a-37d07a52da8b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:05.522843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d943e21a0e6470cbf8812c0cb4f25b3', 'request_id': 'req-b5c6767d-afb5-4514-af3a-37d07a52da8b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:05.561050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] The token cache is being invalidate because project 7d943e21a0e6470cbf8812c0cb4f25b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:05.562514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:05.563123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b5c6767d-afb5-4514-af3a-37d07a52da8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:05.567889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 904/1779] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:05 2022] DELETE /identity/v3/projects/7d943e21a0e6470cbf8812c0cb4f25b3 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:05.574117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ccdb-7172-4c71-a4d9-d268f87e795a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:05.575354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ccdb-7172-4c71-a4d9-d268f87e795a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:05.575354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4ccdb-7172-4c71-a4d9-d268f87e795a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:05.617726 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fbd4ccdb-7172-4c71-a4d9-d268f87e795a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:05.620357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fbd4ccdb-7172-4c71-a4d9-d268f87e795a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:05.768769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 876/1780] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:05.786318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b47e70-d46b-43c1-a1ea-14731c4b5846 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:05.788088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b47e70-d46b-43c1-a1ea-14731c4b5846 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:05.788864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b47e70-d46b-43c1-a1ea-14731c4b5846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:05.852541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-37b47e70-d46b-43c1-a1ea-14731c4b5846 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:05.855924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-37b47e70-d46b-43c1-a1ea-14731c4b5846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:05.902756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 905/1781] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:05.909308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ccdb-7172-4c71-a4d9-d268f87e795a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:05.922233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4ccdb-7172-4c71-a4d9-d268f87e795a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:05.945104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c330f3-f489-4178-b668-8072dd0c911e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:05.946040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c330f3-f489-4178-b668-8072dd0c911e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:05.946502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c330f3-f489-4178-b668-8072dd0c911e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:05.946751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c330f3-f489-4178-b668-8072dd0c911e admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:05.947450 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c330f3-f489-4178-b668-8072dd0c911e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:09:05.947741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c330f3-f489-4178-b668-8072dd0c911e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:05.950149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c330f3-f489-4178-b668-8072dd0c911e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:06.006348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 877/1782] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:09:06.022403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b47e70-d46b-43c1-a1ea-14731c4b5846 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:06.080199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b47e70-d46b-43c1-a1ea-14731c4b5846 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:06.104383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58059-5aa3-4a7a-b27e-7b7d70e5c80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:06.104783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58059-5aa3-4a7a-b27e-7b7d70e5c80b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:06.106538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58059-5aa3-4a7a-b27e-7b7d70e5c80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:06.107103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c58059-5aa3-4a7a-b27e-7b7d70e5c80b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:06.116143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c58059-5aa3-4a7a-b27e-7b7d70e5c80b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:06.120354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c58059-5aa3-4a7a-b27e-7b7d70e5c80b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:06.269494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 906/1783] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:06 2022] POST /identity/v3/projects => generated 387 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:06.282479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c330f3-f489-4178-b668-8072dd0c911e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:06.327071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c330f3-f489-4178-b668-8072dd0c911e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:06.418110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58059-5aa3-4a7a-b27e-7b7d70e5c80b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:06.423191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-15c58059-5aa3-4a7a-b27e-7b7d70e5c80b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:06.542595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497daad1-575d-415b-a5fd-7626b5a28747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:06.543430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497daad1-575d-415b-a5fd-7626b5a28747 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:06.544102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497daad1-575d-415b-a5fd-7626b5a28747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:06.544744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497daad1-575d-415b-a5fd-7626b5a28747 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:06.545584 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497daad1-575d-415b-a5fd-7626b5a28747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:06.549933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497daad1-575d-415b-a5fd-7626b5a28747 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:06.594040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 878/1784] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:06 2022] POST /identity/v3/users => generated 294 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:06.602023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497daad1-575d-415b-a5fd-7626b5a28747 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:06.640852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497daad1-575d-415b-a5fd-7626b5a28747 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:06.649023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7740746-f507-40ec-8206-e8b542e25ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:06.650555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740746-f507-40ec-8206-e8b542e25ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:06.651002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740746-f507-40ec-8206-e8b542e25ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:06.651966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7740746-f507-40ec-8206-e8b542e25ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:06.652850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7740746-f507-40ec-8206-e8b542e25ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:06.654993 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7740746-f507-40ec-8206-e8b542e25ff0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:06.684699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 879/1785] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:06.691197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7740746-f507-40ec-8206-e8b542e25ff0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:06.701332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c58059-5aa3-4a7a-b27e-7b7d70e5c80b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:06.710952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:06.712808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:06.713359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:06.717288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] PATH_INFO: `/v3/projects/9b295a8c0d934c778c94b1ed5f8b1a9d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:06.719338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b295a8c0d934c778c94b1ed5f8b1a9d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:06.721735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7740746-f507-40ec-8206-e8b542e25ff0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:06.725859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:06.734087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ae06c-ac22-4f02-a55a-70281fb9720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:06.735563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ae06c-ac22-4f02-a55a-70281fb9720e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:06.736866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ae06c-ac22-4f02-a55a-70281fb9720e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:06.737482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ae06c-ac22-4f02-a55a-70281fb9720e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:06.738843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ae06c-ac22-4f02-a55a-70281fb9720e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:06.743278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ae06c-ac22-4f02-a55a-70281fb9720e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:06.757391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:06.765252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 880/1786] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:06.771871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ae06c-ac22-4f02-a55a-70281fb9720e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:06.794948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ae06c-ac22-4f02-a55a-70281fb9720e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:06.833161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819e32a1-c5c5-4041-9c25-f9daa0e23d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:06.836798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e32a1-c5c5-4041-9c25-f9daa0e23d50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:06.837333 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e32a1-c5c5-4041-9c25-f9daa0e23d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:06.838068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819e32a1-c5c5-4041-9c25-f9daa0e23d50 admin admin] PATH_INFO: `/v3/projects/5a1aca134743410abd2ddf0530899141/users/fef97a924cc54bb29b074b5c85d82f6c/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:06.888421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e32a1-c5c5-4041-9c25-f9daa0e23d50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1aca134743410abd2ddf0530899141, user_id=fef97a924cc54bb29b074b5c85d82f6c, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:06.891384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:06.902761 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819e32a1-c5c5-4041-9c25-f9daa0e23d50 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:06.920882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b295a8c0d934c778c94b1ed5f8b1a9d', 'request_id': 'req-c2773007-220a-43ba-81fc-955ee7c4ea80'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:06.947770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b295a8c0d934c778c94b1ed5f8b1a9d', 'request_id': 'req-c2773007-220a-43ba-81fc-955ee7c4ea80'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:06.949989 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 881/1787] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:06 2022] PUT /identity/v3/projects/5a1aca134743410abd2ddf0530899141/users/fef97a924cc54bb29b074b5c85d82f6c/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:06.970619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b295a8c0d934c778c94b1ed5f8b1a9d', 'request_id': 'req-c2773007-220a-43ba-81fc-955ee7c4ea80'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:06.977053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee15a10-8655-42b7-977a-928633b40640 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:06.978062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee15a10-8655-42b7-977a-928633b40640 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:06.978839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee15a10-8655-42b7-977a-928633b40640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:07.013673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] The token cache is being invalidate because project 9b295a8c0d934c778c94b1ed5f8b1a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:07.014410 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:07.015844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:07.023770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 907/1788] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:06 2022] DELETE /identity/v3/projects/9b295a8c0d934c778c94b1ed5f8b1a9d => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:07.033492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:07.036239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:07.051189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4ee15a10-8655-42b7-977a-928633b40640 None None] MFA Rules not processed for user `fef97a924cc54bb29b074b5c85d82f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:07.054828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4ee15a10-8655-42b7-977a-928633b40640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:07.220674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 882/1789] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:07.227376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee15a10-8655-42b7-977a-928633b40640 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:07.229307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4ee15a10-8655-42b7-977a-928633b40640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:07.250748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2773007-220a-43ba-81fc-955ee7c4ea80 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:07.421959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee15a10-8655-42b7-977a-928633b40640 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:07.444854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a9ccf-199a-431c-b46a-d791756e2fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:07.445953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a9ccf-199a-431c-b46a-d791756e2fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:07.446726 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a9ccf-199a-431c-b46a-d791756e2fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:07.447156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a9ccf-199a-431c-b46a-d791756e2fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:07.451986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a9ccf-199a-431c-b46a-d791756e2fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:07.455693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a9ccf-199a-431c-b46a-d791756e2fc7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:07.479085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:07.480321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:07.480743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:07.481370 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] PATH_INFO: `/v3/users/7a4a4e9106fd4525a5405d20a19cb559` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:07.483868 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4a4e9106fd4525a5405d20a19cb559)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:07.489373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:07.523682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 883/1790] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:07 2022] POST /identity/v3/projects => generated 387 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:07.535260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a9ccf-199a-431c-b46a-d791756e2fc7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:07.671820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:07.692886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4a4e9106fd4525a5405d20a19cb559'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:07.695281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4a4e9106fd4525a5405d20a19cb559'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:07.698610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4a4e9106fd4525a5405d20a19cb559'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:07.709429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a9ccf-199a-431c-b46a-d791756e2fc7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:07.727800 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4a4e9106fd4525a5405d20a19cb559'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:07.738645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b5b30-b58b-4b40-8093-caafbd729594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:07.742075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b5b30-b58b-4b40-8093-caafbd729594 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:07.744775 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b5b30-b58b-4b40-8093-caafbd729594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:07.745468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6b5b30-b58b-4b40-8093-caafbd729594 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:07.747501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b5b30-b58b-4b40-8093-caafbd729594 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:07.751458 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6b5b30-b58b-4b40-8093-caafbd729594 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:07.754328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4a4e9106fd4525a5405d20a19cb559'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:07.757033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:07.811110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 908/1791] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:07 2022] DELETE /identity/v3/users/7a4a4e9106fd4525a5405d20a19cb559 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:07.811599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 884/1792] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:07 2022] POST /identity/v3/users => generated 300 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:07.812227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:07.813934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:07.903411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ebddd-99c2-4c92-a77c-eb06dab30e83 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:07.909260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d88f4-e3b3-458c-8ef6-08b4cc97b5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:07.910046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d88f4-e3b3-458c-8ef6-08b4cc97b5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:07.910440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d88f4-e3b3-458c-8ef6-08b4cc97b5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:07.910789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d88f4-e3b3-458c-8ef6-08b4cc97b5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:07.911706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d88f4-e3b3-458c-8ef6-08b4cc97b5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:07.915039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d88f4-e3b3-458c-8ef6-08b4cc97b5c8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:07.933927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 909/1793] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:07.935131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b5b30-b58b-4b40-8093-caafbd729594 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:07.957239 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6b5b30-b58b-4b40-8093-caafbd729594 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:07.971556 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb4a14-7351-4ff6-ad1d-97a4c068bda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:07.973615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb4a14-7351-4ff6-ad1d-97a4c068bda2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:07.975834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb4a14-7351-4ff6-ad1d-97a4c068bda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:07.977134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb4a14-7351-4ff6-ad1d-97a4c068bda2 admin admin] PATH_INFO: `/v3/projects/21049b0ee7cb468e857d108fadbbcee3/users/b71aa1d4fb17438e9fb2c667729a5bee/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:08.016873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb4a14-7351-4ff6-ad1d-97a4c068bda2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21049b0ee7cb468e857d108fadbbcee3, user_id=b71aa1d4fb17438e9fb2c667729a5bee, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:08.025366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb4a14-7351-4ff6-ad1d-97a4c068bda2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:08.088976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 885/1794] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:07 2022] PUT /identity/v3/projects/21049b0ee7cb468e857d108fadbbcee3/users/b71aa1d4fb17438e9fb2c667729a5bee/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:08.106053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d88f4-e3b3-458c-8ef6-08b4cc97b5c8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:08.127713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d88f4-e3b3-458c-8ef6-08b4cc97b5c8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:08.333222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac9bb50-3ade-4096-a93f-f0f75fb96d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:08.335871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9bb50-3ade-4096-a93f-f0f75fb96d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:08.336617 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9bb50-3ade-4096-a93f-f0f75fb96d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:08.339999 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac9bb50-3ade-4096-a93f-f0f75fb96d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:08.343367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9bb50-3ade-4096-a93f-f0f75fb96d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:08.355208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac9bb50-3ade-4096-a93f-f0f75fb96d0e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:08.403110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 910/1795] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:08.404313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb4a14-7351-4ff6-ad1d-97a4c068bda2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:08.419479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb4a14-7351-4ff6-ad1d-97a4c068bda2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:08.436878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc3f07-9d51-47b0-8c26-963f1798a140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:08.437603 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc3f07-9d51-47b0-8c26-963f1798a140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:08.438009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc3f07-9d51-47b0-8c26-963f1798a140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:08.438215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc3f07-9d51-47b0-8c26-963f1798a140 admin admin] PATH_INFO: `/v3/projects/21049b0ee7cb468e857d108fadbbcee3/users/b71aa1d4fb17438e9fb2c667729a5bee/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:08.440441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc3f07-9d51-47b0-8c26-963f1798a140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21049b0ee7cb468e857d108fadbbcee3, user_id=b71aa1d4fb17438e9fb2c667729a5bee, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:08.444944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc3f07-9d51-47b0-8c26-963f1798a140 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:08.484488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 886/1796] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:08 2022] PUT /identity/v3/projects/21049b0ee7cb468e857d108fadbbcee3/users/b71aa1d4fb17438e9fb2c667729a5bee/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:08.497833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66942202-cc0c-4ec6-b034-ccbaa3326b3a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:08.498453 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66942202-cc0c-4ec6-b034-ccbaa3326b3a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:08.498878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66942202-cc0c-4ec6-b034-ccbaa3326b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:08.579383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-66942202-cc0c-4ec6-b034-ccbaa3326b3a None None] MFA Rules not processed for user `b71aa1d4fb17438e9fb2c667729a5bee`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:08.587045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-66942202-cc0c-4ec6-b034-ccbaa3326b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:08.610055 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc3f07-9d51-47b0-8c26-963f1798a140 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:08.613498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a3fc3f07-9d51-47b0-8c26-963f1798a140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:08.691374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 911/1797] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:08.705485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66942202-cc0c-4ec6-b034-ccbaa3326b3a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:08.709393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-66942202-cc0c-4ec6-b034-ccbaa3326b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:08.729056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:08.783405 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc3f07-9d51-47b0-8c26-963f1798a140 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:08.794204 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:08.794921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:08.795153 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:08.795454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] PATH_INFO: `/v3/projects/835ecb3974e14787bc16fa31a4e8a640` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:08.796972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=835ecb3974e14787bc16fa31a4e8a640)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:08.799113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:08.828897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66942202-cc0c-4ec6-b034-ccbaa3326b3a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:08.835341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25ab81-19ff-4b99-b1f7-a1cd512728de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:08.836153 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25ab81-19ff-4b99-b1f7-a1cd512728de service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:08.836560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25ab81-19ff-4b99-b1f7-a1cd512728de service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:08.836936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25ab81-19ff-4b99-b1f7-a1cd512728de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:08.851850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25ab81-19ff-4b99-b1f7-a1cd512728de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:08.853621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25ab81-19ff-4b99-b1f7-a1cd512728de service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:08.867540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 912/1798] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:08.880817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '835ecb3974e14787bc16fa31a4e8a640', 'request_id': 'req-e42cb083-6352-43d2-b440-4f3d086c62e8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:08.892312 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '835ecb3974e14787bc16fa31a4e8a640', 'request_id': 'req-e42cb083-6352-43d2-b440-4f3d086c62e8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:08.900416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '835ecb3974e14787bc16fa31a4e8a640', 'request_id': 'req-e42cb083-6352-43d2-b440-4f3d086c62e8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:08.928385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] The token cache is being invalidate because project 835ecb3974e14787bc16fa31a4e8a640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:08.929318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:08.930643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:08.935758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 887/1799] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:08 2022] DELETE /identity/v3/projects/835ecb3974e14787bc16fa31a4e8a640 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:08.937489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a7b93-8c89-4f07-931a-c63e97d1cdf5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:08.937796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a7b93-8c89-4f07-931a-c63e97d1cdf5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:08.938217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a7b93-8c89-4f07-931a-c63e97d1cdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:08.966302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d78a7b93-8c89-4f07-931a-c63e97d1cdf5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:08.967706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d78a7b93-8c89-4f07-931a-c63e97d1cdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:09.001809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:09.006068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:09.065828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 913/1800] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:09.076010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6dfb6-37ce-4ef8-b9ca-7342a78780ad None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.076300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6dfb6-37ce-4ef8-b9ca-7342a78780ad None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.076571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6dfb6-37ce-4ef8-b9ca-7342a78780ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.088762 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:09.108495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c7c6dfb6-37ce-4ef8-b9ca-7342a78780ad None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:09.110856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c7c6dfb6-37ce-4ef8-b9ca-7342a78780ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:09.124186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 914/1801] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:09.131481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6dfb6-37ce-4ef8-b9ca-7342a78780ad None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:09.131689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cb083-6352-43d2-b440-4f3d086c62e8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:09.136984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:09.137748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.138135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.138494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] PATH_INFO: `/v3/users/fef97a924cc54bb29b074b5c85d82f6c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.140001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef97a924cc54bb29b074b5c85d82f6c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:09.142164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:09.155842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6dfb6-37ce-4ef8-b9ca-7342a78780ad None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:09.164351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0fc38-e940-4c15-a5aa-1048b952253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:09.165361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0fc38-e940-4c15-a5aa-1048b952253e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.165876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0fc38-e940-4c15-a5aa-1048b952253e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.166351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed0fc38-e940-4c15-a5aa-1048b952253e admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.167490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0fc38-e940-4c15-a5aa-1048b952253e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:09:09.169413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0fc38-e940-4c15-a5aa-1048b952253e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:09.171632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed0fc38-e940-4c15-a5aa-1048b952253e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:09.204194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 915/1802] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:09:09.205495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0fc38-e940-4c15-a5aa-1048b952253e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:09.227405 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed0fc38-e940-4c15-a5aa-1048b952253e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:09.229329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef97a924cc54bb29b074b5c85d82f6c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:09.236123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d196c1-310a-40da-9c67-90b3d239171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:09.237675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d196c1-310a-40da-9c67-90b3d239171b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.238307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d196c1-310a-40da-9c67-90b3d239171b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.239148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d196c1-310a-40da-9c67-90b3d239171b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.240591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef97a924cc54bb29b074b5c85d82f6c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:09.241312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d196c1-310a-40da-9c67-90b3d239171b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:09.241734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef97a924cc54bb29b074b5c85d82f6c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:09.242611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef97a924cc54bb29b074b5c85d82f6c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:09.244903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d196c1-310a-40da-9c67-90b3d239171b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:09.251519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-94e9cd1f-266d-4897-a05a-820c8b5d4b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef97a924cc54bb29b074b5c85d82f6c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:09.278934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 888/1803] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:08 2022] DELETE /identity/v3/users/fef97a924cc54bb29b074b5c85d82f6c => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:09.281243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef670d3-384f-4b3e-9cff-203521dee2b9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.281708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef670d3-384f-4b3e-9cff-203521dee2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.282189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef670d3-384f-4b3e-9cff-203521dee2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.306080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7ef670d3-384f-4b3e-9cff-203521dee2b9 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:09.307424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7ef670d3-384f-4b3e-9cff-203521dee2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:09.310170 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 916/1804] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:09 2022] POST /identity/v3/projects => generated 369 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:09.311346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d196c1-310a-40da-9c67-90b3d239171b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:09.314139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-81d196c1-310a-40da-9c67-90b3d239171b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:09.461154 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 889/1805] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:09.462018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d196c1-310a-40da-9c67-90b3d239171b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:09.475345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef670d3-384f-4b3e-9cff-203521dee2b9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:09.477436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7ef670d3-384f-4b3e-9cff-203521dee2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:09.482037 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e31898-660a-40f7-a722-86b563a0b8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:09.482918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e31898-660a-40f7-a722-86b563a0b8c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.482918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e31898-660a-40f7-a722-86b563a0b8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.483204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e31898-660a-40f7-a722-86b563a0b8c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.483850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e31898-660a-40f7-a722-86b563a0b8c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:09.485898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e31898-660a-40f7-a722-86b563a0b8c2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:09.512557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:09.527940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 917/1806] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:09 2022] POST /identity/v3/users => generated 285 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:09.529629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e31898-660a-40f7-a722-86b563a0b8c2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:09.543318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e31898-660a-40f7-a722-86b563a0b8c2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:09.548865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3109b-e4f0-4442-80d7-c996968a1431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:09.549591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3109b-e4f0-4442-80d7-c996968a1431 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.549990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3109b-e4f0-4442-80d7-c996968a1431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.550324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3109b-e4f0-4442-80d7-c996968a1431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.551003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3109b-e4f0-4442-80d7-c996968a1431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:09.553245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3109b-e4f0-4442-80d7-c996968a1431 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:09.571377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 918/1807] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:09.572387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3109b-e4f0-4442-80d7-c996968a1431 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:09.588640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3109b-e4f0-4442-80d7-c996968a1431 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:09.595128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3efc6-735f-4fa5-b3a8-97ba1c11891c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:09.595774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3efc6-735f-4fa5-b3a8-97ba1c11891c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.596053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3efc6-735f-4fa5-b3a8-97ba1c11891c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.596473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3efc6-735f-4fa5-b3a8-97ba1c11891c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.597140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3efc6-735f-4fa5-b3a8-97ba1c11891c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:09.598907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3efc6-735f-4fa5-b3a8-97ba1c11891c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:09.611389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef670d3-384f-4b3e-9cff-203521dee2b9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:09.614125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 919/1808] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:09.619585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3efc6-735f-4fa5-b3a8-97ba1c11891c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:09.619896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662df50d-8c3c-4b07-8dbb-671b7bb01369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:09.620834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df50d-8c3c-4b07-8dbb-671b7bb01369 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.621363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df50d-8c3c-4b07-8dbb-671b7bb01369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.621867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662df50d-8c3c-4b07-8dbb-671b7bb01369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.637032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:09.639076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3efc6-735f-4fa5-b3a8-97ba1c11891c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:09.642480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662df50d-8c3c-4b07-8dbb-671b7bb01369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:09.646507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662df50d-8c3c-4b07-8dbb-671b7bb01369 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:09.651806 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36f3b9-0c02-430d-8601-21330f8fdfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:09.653169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36f3b9-0c02-430d-8601-21330f8fdfef admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.653606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36f3b9-0c02-430d-8601-21330f8fdfef admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.654188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b36f3b9-0c02-430d-8601-21330f8fdfef admin admin] PATH_INFO: `/v3/projects/15ea79d7c6114867aeb2a6ffdba2ff44/users/d0df6591f1924fffbbc3e2d4702cd6c8/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.678311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 890/1809] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:09:09.697823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36f3b9-0c02-430d-8601-21330f8fdfef admin admin] RBAC: Authorizing `identity:create_grant(project_id=15ea79d7c6114867aeb2a6ffdba2ff44, user_id=d0df6591f1924fffbbc3e2d4702cd6c8, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:09.697823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b36f3b9-0c02-430d-8601-21330f8fdfef admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:09.722712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 920/1810] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:09 2022] PUT /identity/v3/projects/15ea79d7c6114867aeb2a6ffdba2ff44/users/d0df6591f1924fffbbc3e2d4702cd6c8/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:09.734488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8caee-fc6b-4255-8e33-18e3de00b2b2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.734757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8caee-fc6b-4255-8e33-18e3de00b2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.735038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d8caee-fc6b-4255-8e33-18e3de00b2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.783887 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-95d8caee-fc6b-4255-8e33-18e3de00b2b2 None None] MFA Rules not processed for user `d0df6591f1924fffbbc3e2d4702cd6c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:09.784868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-95d8caee-fc6b-4255-8e33-18e3de00b2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:09.884060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 891/1811] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:09.889461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36f3b9-0c02-430d-8601-21330f8fdfef admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:09.985234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b36f3b9-0c02-430d-8601-21330f8fdfef admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:09.992352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb0077-159f-4f78-8270-9d26b00c8646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:09.992994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb0077-159f-4f78-8270-9d26b00c8646 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:09.993305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb0077-159f-4f78-8270-9d26b00c8646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:09.993607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fb0077-159f-4f78-8270-9d26b00c8646 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:09.994233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb0077-159f-4f78-8270-9d26b00c8646 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:09.996320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fb0077-159f-4f78-8270-9d26b00c8646 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:10.037555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 921/1812] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:09 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:10.041342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8caee-fc6b-4255-8e33-18e3de00b2b2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:10.054071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d8caee-fc6b-4255-8e33-18e3de00b2b2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:10.133531 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fa65a-794c-41fe-8507-909c69a4891c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:10.134211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fa65a-794c-41fe-8507-909c69a4891c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:10.134639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fa65a-794c-41fe-8507-909c69a4891c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:10.134951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40fa65a-794c-41fe-8507-909c69a4891c admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:10.135600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fa65a-794c-41fe-8507-909c69a4891c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:10.137616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40fa65a-794c-41fe-8507-909c69a4891c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:10.165459 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 892/1813] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:10 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:10.166903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb0077-159f-4f78-8270-9d26b00c8646 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:10.180954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fb0077-159f-4f78-8270-9d26b00c8646 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:10.190543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e24a1-4fe7-4f68-a7cb-37edc66a9afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:10.191420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e24a1-4fe7-4f68-a7cb-37edc66a9afd admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:10.191854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e24a1-4fe7-4f68-a7cb-37edc66a9afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:10.192241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e24a1-4fe7-4f68-a7cb-37edc66a9afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:10.193061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e24a1-4fe7-4f68-a7cb-37edc66a9afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:10.195877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e24a1-4fe7-4f68-a7cb-37edc66a9afd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:10.212862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 922/1814] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:10.222337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fa65a-794c-41fe-8507-909c69a4891c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:10.247322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40fa65a-794c-41fe-8507-909c69a4891c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:10.257112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53bc75-f22e-4c71-940c-5dd878f0793f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:10.258855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53bc75-f22e-4c71-940c-5dd878f0793f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:10.260309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53bc75-f22e-4c71-940c-5dd878f0793f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:10.260309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53bc75-f22e-4c71-940c-5dd878f0793f admin admin] PATH_INFO: `/v3/projects/8f6b5517c1944e47b1a04105676c3503/users/bcf75a8843894daeb1a920e25fdc8ccb/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:10.295868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53bc75-f22e-4c71-940c-5dd878f0793f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6b5517c1944e47b1a04105676c3503, user_id=bcf75a8843894daeb1a920e25fdc8ccb, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:10.299521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53bc75-f22e-4c71-940c-5dd878f0793f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:10.323039 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 893/1815] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:10 2022] PUT /identity/v3/projects/8f6b5517c1944e47b1a04105676c3503/users/bcf75a8843894daeb1a920e25fdc8ccb/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:10.329175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e24a1-4fe7-4f68-a7cb-37edc66a9afd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:10.347895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e24a1-4fe7-4f68-a7cb-37edc66a9afd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:10.466696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb93e3-bf2a-4561-830c-461ea188c60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:10.467732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb93e3-bf2a-4561-830c-461ea188c60a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:10.468793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb93e3-bf2a-4561-830c-461ea188c60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:10.469275 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb93e3-bf2a-4561-830c-461ea188c60a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:10.470491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb93e3-bf2a-4561-830c-461ea188c60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:10.474325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb93e3-bf2a-4561-830c-461ea188c60a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:10.502966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53bc75-f22e-4c71-940c-5dd878f0793f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:10.506797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ab53bc75-f22e-4c71-940c-5dd878f0793f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:10.514069 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 923/1816] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:10.518342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb93e3-bf2a-4561-830c-461ea188c60a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:10.531855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb93e3-bf2a-4561-830c-461ea188c60a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:10.538688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2552467-9701-423a-ab99-b943cf46b321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:10.539575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2552467-9701-423a-ab99-b943cf46b321 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:10.539895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2552467-9701-423a-ab99-b943cf46b321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:10.540217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2552467-9701-423a-ab99-b943cf46b321 admin admin] PATH_INFO: `/v3/projects/8f6b5517c1944e47b1a04105676c3503/users/bcf75a8843894daeb1a920e25fdc8ccb/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:10.542485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2552467-9701-423a-ab99-b943cf46b321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6b5517c1944e47b1a04105676c3503, user_id=bcf75a8843894daeb1a920e25fdc8ccb, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:10.547013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2552467-9701-423a-ab99-b943cf46b321 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:10.555568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53bc75-f22e-4c71-940c-5dd878f0793f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:10.562887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 924/1817] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:10 2022] PUT /identity/v3/projects/8f6b5517c1944e47b1a04105676c3503/users/bcf75a8843894daeb1a920e25fdc8ccb/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:10.581923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673188-f24f-4f4c-b3cf-7b3d23641566 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:10.582382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673188-f24f-4f4c-b3cf-7b3d23641566 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:10.582723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c673188-f24f-4f4c-b3cf-7b3d23641566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:10.612022 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:10.626216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8c673188-f24f-4f4c-b3cf-7b3d23641566 None None] MFA Rules not processed for user `bcf75a8843894daeb1a920e25fdc8ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:10.628044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8c673188-f24f-4f4c-b3cf-7b3d23641566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:10.647650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:10.665421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:10.666615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:10.667328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:10.668618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] PATH_INFO: `/v3/projects/5a1aca134743410abd2ddf0530899141` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:10.670419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1aca134743410abd2ddf0530899141)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:10.673975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:10.741287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 925/1818] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:10 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:10.758791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1aca134743410abd2ddf0530899141', 'request_id': 'req-a6d25ee4-4d03-49ee-ac92-f02593284936'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:10.759334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c673188-f24f-4f4c-b3cf-7b3d23641566 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:10.761453 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8c673188-f24f-4f4c-b3cf-7b3d23641566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:10.770670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1aca134743410abd2ddf0530899141', 'request_id': 'req-a6d25ee4-4d03-49ee-ac92-f02593284936'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:10.779897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1aca134743410abd2ddf0530899141', 'request_id': 'req-a6d25ee4-4d03-49ee-ac92-f02593284936'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:10.809446 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] The token cache is being invalidate because project 5a1aca134743410abd2ddf0530899141 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:10.810108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:10.811249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:10.817863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 894/1819] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:10 2022] DELETE /identity/v3/projects/5a1aca134743410abd2ddf0530899141 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:10.821372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:10.823903 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:10.867683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c673188-f24f-4f4c-b3cf-7b3d23641566 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:10.914166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d25ee4-4d03-49ee-ac92-f02593284936 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:10.920021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:10.921173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:10.921173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:10.921446 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] PATH_INFO: `/v3/users/b71aa1d4fb17438e9fb2c667729a5bee` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:10.922861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b71aa1d4fb17438e9fb2c667729a5bee)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:10.925013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:10.948439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f0f7-af7c-4f45-948f-2709438890e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:10.949222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f0f7-af7c-4f45-948f-2709438890e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:10.949572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f0f7-af7c-4f45-948f-2709438890e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:10.949958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43f0f7-af7c-4f45-948f-2709438890e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:10.952274 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6a43f0f7-af7c-4f45-948f-2709438890e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:11.008522 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b71aa1d4fb17438e9fb2c667729a5bee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:11.030321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b71aa1d4fb17438e9fb2c667729a5bee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:11.032722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b71aa1d4fb17438e9fb2c667729a5bee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:11.033650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b71aa1d4fb17438e9fb2c667729a5bee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:11.044285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b71aa1d4fb17438e9fb2c667729a5bee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:11.069804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:11.078423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 895/1820] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:10 2022] DELETE /identity/v3/users/b71aa1d4fb17438e9fb2c667729a5bee => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:11.093229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43f0f7-af7c-4f45-948f-2709438890e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:11.094908 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43f0f7-af7c-4f45-948f-2709438890e4 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:11.101860 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:11.108100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 926/1821] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:11.165066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:11.166946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:11.247179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9b288-7e45-470d-9cc9-093a3dc629d4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:11.254886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ab64f-227d-41e7-a55a-5afc4c53cc30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:11.254886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ab64f-227d-41e7-a55a-5afc4c53cc30 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:11.254886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ab64f-227d-41e7-a55a-5afc4c53cc30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:11.254886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ab64f-227d-41e7-a55a-5afc4c53cc30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:11.258029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a58ab64f-227d-41e7-a55a-5afc4c53cc30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:11.340848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ab64f-227d-41e7-a55a-5afc4c53cc30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:11.342955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ab64f-227d-41e7-a55a-5afc4c53cc30 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:11.354555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 896/1822] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:11.452160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f0f7-af7c-4f45-948f-2709438890e4 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:11.468632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43f0f7-af7c-4f45-948f-2709438890e4 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:11.583386 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:11.585236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:11.585686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:11.586030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] PATH_INFO: `/v3/projects/21049b0ee7cb468e857d108fadbbcee3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:11.588061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21049b0ee7cb468e857d108fadbbcee3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:11.591592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:11.697731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21049b0ee7cb468e857d108fadbbcee3', 'request_id': 'req-b676a200-62ac-42f9-8624-dcdec8cdfa45'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:11.715390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21049b0ee7cb468e857d108fadbbcee3', 'request_id': 'req-b676a200-62ac-42f9-8624-dcdec8cdfa45'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:11.730076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21049b0ee7cb468e857d108fadbbcee3', 'request_id': 'req-b676a200-62ac-42f9-8624-dcdec8cdfa45'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:11.757267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] The token cache is being invalidate because project 21049b0ee7cb468e857d108fadbbcee3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:11.758403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:11.759879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b676a200-62ac-42f9-8624-dcdec8cdfa45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:11.768238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 927/1823] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:11 2022] DELETE /identity/v3/projects/21049b0ee7cb468e857d108fadbbcee3 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:11.773731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2a1bc-ebc5-4687-84c5-11c9b1e509e5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:11.774511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2a1bc-ebc5-4687-84c5-11c9b1e509e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:11.774999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e2a1bc-ebc5-4687-84c5-11c9b1e509e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:11.829561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-72e2a1bc-ebc5-4687-84c5-11c9b1e509e5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:11.831330 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-72e2a1bc-ebc5-4687-84c5-11c9b1e509e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:11.922899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 897/1824] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:11.934952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d796ef-5797-4cee-9c45-76d7ef987d5c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:11.935398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d796ef-5797-4cee-9c45-76d7ef987d5c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:11.935871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d796ef-5797-4cee-9c45-76d7ef987d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:11.962443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e3d796ef-5797-4cee-9c45-76d7ef987d5c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:11.963929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e3d796ef-5797-4cee-9c45-76d7ef987d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:11.975243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 928/1825] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:11.980571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2a1bc-ebc5-4687-84c5-11c9b1e509e5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:11.992696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e2a1bc-ebc5-4687-84c5-11c9b1e509e5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:11.998184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dee64-e432-4015-97fc-5e43930c9351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:11.998829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dee64-e432-4015-97fc-5e43930c9351 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:11.999110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dee64-e432-4015-97fc-5e43930c9351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:11.999389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dee64-e432-4015-97fc-5e43930c9351 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.000072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dee64-e432-4015-97fc-5e43930c9351 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:09:12.000382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dee64-e432-4015-97fc-5e43930c9351 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:12.002185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dee64-e432-4015-97fc-5e43930c9351 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:12.021783 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 898/1826] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:09:12.023596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d796ef-5797-4cee-9c45-76d7ef987d5c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:12.034751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d796ef-5797-4cee-9c45-76d7ef987d5c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:12.041226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00244e-3566-4fcc-a37e-16afa1ec5481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:12.041900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00244e-3566-4fcc-a37e-16afa1ec5481 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:12.042183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00244e-3566-4fcc-a37e-16afa1ec5481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:12.042452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db00244e-3566-4fcc-a37e-16afa1ec5481 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.043100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00244e-3566-4fcc-a37e-16afa1ec5481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:12.045090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db00244e-3566-4fcc-a37e-16afa1ec5481 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:12.079523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 929/1827] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:12 2022] POST /identity/v3/projects => generated 399 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:12.085246 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dee64-e432-4015-97fc-5e43930c9351 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:12.104867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dee64-e432-4015-97fc-5e43930c9351 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:12.180288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:12.204545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df766c2-2053-4797-b705-1f9cdefef16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:12.205277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df766c2-2053-4797-b705-1f9cdefef16e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:12.205564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df766c2-2053-4797-b705-1f9cdefef16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:12.205877 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df766c2-2053-4797-b705-1f9cdefef16e admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.206541 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df766c2-2053-4797-b705-1f9cdefef16e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:12.208933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df766c2-2053-4797-b705-1f9cdefef16e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:12.234332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 899/1828] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:12 2022] POST /identity/v3/users => generated 300 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:12.238809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00244e-3566-4fcc-a37e-16afa1ec5481 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:12.252618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db00244e-3566-4fcc-a37e-16afa1ec5481 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:12.259182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df6947-319b-4585-b1c6-749240302c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:12.259937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df6947-319b-4585-b1c6-749240302c3d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:12.260393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df6947-319b-4585-b1c6-749240302c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:12.260839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7df6947-319b-4585-b1c6-749240302c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.261698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df6947-319b-4585-b1c6-749240302c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:12.263571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7df6947-319b-4585-b1c6-749240302c3d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:12.276365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 930/1829] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:09:12.282051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df766c2-2053-4797-b705-1f9cdefef16e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:12.301369 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df766c2-2053-4797-b705-1f9cdefef16e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:12.307914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c879091-5539-4bef-8bf1-2b8702f28c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:12.308402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c879091-5539-4bef-8bf1-2b8702f28c6d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:12.308833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c879091-5539-4bef-8bf1-2b8702f28c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:12.309125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c879091-5539-4bef-8bf1-2b8702f28c6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.310006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c879091-5539-4bef-8bf1-2b8702f28c6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:12.311693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c879091-5539-4bef-8bf1-2b8702f28c6d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:12.329860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 900/1830] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:12.331756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df6947-319b-4585-b1c6-749240302c3d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:12.346786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7df6947-319b-4585-b1c6-749240302c3d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:12.355840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aeb46a-196d-4073-8539-313e96dc0f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:12.357043 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeb46a-196d-4073-8539-313e96dc0f1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:12.357542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeb46a-196d-4073-8539-313e96dc0f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:12.358080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aeb46a-196d-4073-8539-313e96dc0f1e admin admin] PATH_INFO: `/v3/projects/86c3095b2979482c935bd3f32c5d7ec3/users/a6548b2c559a4db5ad01386db2bb9080/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.400609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aeb46a-196d-4073-8539-313e96dc0f1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c3095b2979482c935bd3f32c5d7ec3, user_id=a6548b2c559a4db5ad01386db2bb9080, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:12.406172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aeb46a-196d-4073-8539-313e96dc0f1e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:12.427261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 931/1831] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:12 2022] PUT /identity/v3/projects/86c3095b2979482c935bd3f32c5d7ec3/users/a6548b2c559a4db5ad01386db2bb9080/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:12.446898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dacaf-089e-4621-b375-c34eabe8a574 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:12.447215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dacaf-089e-4621-b375-c34eabe8a574 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:12.447498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9dacaf-089e-4621-b375-c34eabe8a574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.487489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ef9dacaf-089e-4621-b375-c34eabe8a574 None None] MFA Rules not processed for user `a6548b2c559a4db5ad01386db2bb9080`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:12.488717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ef9dacaf-089e-4621-b375-c34eabe8a574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:12.588135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 901/1832] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:12 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:12.595595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aeb46a-196d-4073-8539-313e96dc0f1e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:12.653217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:12.692169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aeb46a-196d-4073-8539-313e96dc0f1e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:12.697021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591437d6-798a-4b4d-9793-1df77a9f2e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:12.697646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591437d6-798a-4b4d-9793-1df77a9f2e93 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:12.697959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591437d6-798a-4b4d-9793-1df77a9f2e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:12.698239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591437d6-798a-4b4d-9793-1df77a9f2e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.698882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591437d6-798a-4b4d-9793-1df77a9f2e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:12.700844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591437d6-798a-4b4d-9793-1df77a9f2e93 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:12.732767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 932/1833] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:12 2022] POST /identity/v3/projects => generated 401 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:12.737333 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dacaf-089e-4621-b375-c34eabe8a574 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:12.751761 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9dacaf-089e-4621-b375-c34eabe8a574 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:12.820223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4cf82-468a-4a72-ba76-8daa98550778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:12.820848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4cf82-468a-4a72-ba76-8daa98550778 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:12.821197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4cf82-468a-4a72-ba76-8daa98550778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:12.821510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4cf82-468a-4a72-ba76-8daa98550778 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.822128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4cf82-468a-4a72-ba76-8daa98550778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:12.824155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4cf82-468a-4a72-ba76-8daa98550778 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:12.849526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 902/1834] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:12 2022] POST /identity/v3/users => generated 307 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:12.855870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591437d6-798a-4b4d-9793-1df77a9f2e93 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:12.875373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591437d6-798a-4b4d-9793-1df77a9f2e93 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:12.883996 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba5da5-7da0-4273-800e-8211ba9246a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:12.884793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba5da5-7da0-4273-800e-8211ba9246a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:12.885127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba5da5-7da0-4273-800e-8211ba9246a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:12.885127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ba5da5-7da0-4273-800e-8211ba9246a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.886137 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba5da5-7da0-4273-800e-8211ba9246a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:12.888603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ba5da5-7da0-4273-800e-8211ba9246a4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:12.913073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 933/1835] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:12.919653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4cf82-468a-4a72-ba76-8daa98550778 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:12.932725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4cf82-468a-4a72-ba76-8daa98550778 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:12.939230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64a1f2-dafa-46e5-acc9-201169322488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:12.939778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64a1f2-dafa-46e5-acc9-201169322488 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:12.940058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64a1f2-dafa-46e5-acc9-201169322488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:12.940376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64a1f2-dafa-46e5-acc9-201169322488 admin admin] PATH_INFO: `/v3/projects/38b56d4fd326438c9cff1414f88f93c5/users/2795e9eb58f84df680174877884acd7a/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:12.961019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64a1f2-dafa-46e5-acc9-201169322488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b56d4fd326438c9cff1414f88f93c5, user_id=2795e9eb58f84df680174877884acd7a, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:12.968742 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64a1f2-dafa-46e5-acc9-201169322488 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:12.992886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 903/1836] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:12 2022] PUT /identity/v3/projects/38b56d4fd326438c9cff1414f88f93c5/users/2795e9eb58f84df680174877884acd7a/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:12.998251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba5da5-7da0-4273-800e-8211ba9246a4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:13.028045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ba5da5-7da0-4273-800e-8211ba9246a4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:13.139687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc12d0f-c21e-4882-b3fa-cb89080ac522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:13.140615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc12d0f-c21e-4882-b3fa-cb89080ac522 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:13.141395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc12d0f-c21e-4882-b3fa-cb89080ac522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:13.141909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc12d0f-c21e-4882-b3fa-cb89080ac522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:13.142954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc12d0f-c21e-4882-b3fa-cb89080ac522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:13.144817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc12d0f-c21e-4882-b3fa-cb89080ac522 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:13.159774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 934/1837] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:13.164758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64a1f2-dafa-46e5-acc9-201169322488 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:13.187358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64a1f2-dafa-46e5-acc9-201169322488 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:13.193671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6ccc7-5bbf-4f27-91fe-8ce50a90269b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:13.194649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6ccc7-5bbf-4f27-91fe-8ce50a90269b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:13.194931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6ccc7-5bbf-4f27-91fe-8ce50a90269b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:13.195208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6ccc7-5bbf-4f27-91fe-8ce50a90269b admin admin] PATH_INFO: `/v3/projects/38b56d4fd326438c9cff1414f88f93c5/users/2795e9eb58f84df680174877884acd7a/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:13.197642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6ccc7-5bbf-4f27-91fe-8ce50a90269b admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b56d4fd326438c9cff1414f88f93c5, user_id=2795e9eb58f84df680174877884acd7a, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:13.200530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6ccc7-5bbf-4f27-91fe-8ce50a90269b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:13.212688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 904/1838] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:13 2022] PUT /identity/v3/projects/38b56d4fd326438c9cff1414f88f93c5/users/2795e9eb58f84df680174877884acd7a/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:13.223880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5754a8-f1cb-467c-a755-bf138a7eb8b8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:13.224201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5754a8-f1cb-467c-a755-bf138a7eb8b8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:13.225018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5754a8-f1cb-467c-a755-bf138a7eb8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:13.266596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5f5754a8-f1cb-467c-a755-bf138a7eb8b8 None None] MFA Rules not processed for user `2795e9eb58f84df680174877884acd7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:13.268985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5f5754a8-f1cb-467c-a755-bf138a7eb8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:13.422739 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 935/1839] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:13 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:13.438871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6ccc7-5bbf-4f27-91fe-8ce50a90269b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:13.441927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0bb6ccc7-5bbf-4f27-91fe-8ce50a90269b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:13.460582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:13.534876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6ccc7-5bbf-4f27-91fe-8ce50a90269b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:13.544675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ad449-0355-4492-bbe8-11689821a5c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:13.546106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ad449-0355-4492-bbe8-11689821a5c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:13.546891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ad449-0355-4492-bbe8-11689821a5c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:13.547530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ad449-0355-4492-bbe8-11689821a5c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:13.571320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ad449-0355-4492-bbe8-11689821a5c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:13.575174 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ad449-0355-4492-bbe8-11689821a5c3 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:13.601310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 905/1840] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:13 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:13.715460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5754a8-f1cb-467c-a755-bf138a7eb8b8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:13.736343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5754a8-f1cb-467c-a755-bf138a7eb8b8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:13.742580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d1c13-9b00-432e-93ea-c9ab3d35ed12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:13.743272 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d1c13-9b00-432e-93ea-c9ab3d35ed12 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:13.746307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d1c13-9b00-432e-93ea-c9ab3d35ed12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:13.746307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d1c13-9b00-432e-93ea-c9ab3d35ed12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:13.781261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:13.839164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d1c13-9b00-432e-93ea-c9ab3d35ed12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:13.841675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d1c13-9b00-432e-93ea-c9ab3d35ed12 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:13.853303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 936/1841] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:13 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:15.419287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ad449-0355-4492-bbe8-11689821a5c3 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:15.421744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ee8ad449-0355-4492-bbe8-11689821a5c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:15.566398 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ad449-0355-4492-bbe8-11689821a5c3 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:15.572613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:15.573294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:15.573586 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:15.573868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] PATH_INFO: `/v3/users/d0df6591f1924fffbbc3e2d4702cd6c8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:15.575164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0df6591f1924fffbbc3e2d4702cd6c8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:15.577329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:15.669690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0df6591f1924fffbbc3e2d4702cd6c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:15.691386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0df6591f1924fffbbc3e2d4702cd6c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:15.692608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0df6591f1924fffbbc3e2d4702cd6c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:15.693722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0df6591f1924fffbbc3e2d4702cd6c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:15.711505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0df6591f1924fffbbc3e2d4702cd6c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:15.749601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 906/1842] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:15 2022] DELETE /identity/v3/users/d0df6591f1924fffbbc3e2d4702cd6c8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:15.755434 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96d527-a577-4881-82f6-d5592ff99541 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:15.756038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96d527-a577-4881-82f6-d5592ff99541 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:15.756455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc96d527-a577-4881-82f6-d5592ff99541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:15.811386 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cc96d527-a577-4881-82f6-d5592ff99541 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:15.817132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cc96d527-a577-4881-82f6-d5592ff99541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:15.932341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 937/1843] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:15.944121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:15.945485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:15.984147 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:16.026681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9a5b0-bb30-4c0d-ab1c-4caa53134580 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:16.031144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5f5e3-5794-4be3-9ec3-bd10e4856a39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:16.031917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5f5e3-5794-4be3-9ec3-bd10e4856a39 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:16.032635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5f5e3-5794-4be3-9ec3-bd10e4856a39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:16.033038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea5f5e3-5794-4be3-9ec3-bd10e4856a39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:16.045292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5f5e3-5794-4be3-9ec3-bd10e4856a39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:16.048061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea5f5e3-5794-4be3-9ec3-bd10e4856a39 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:16.072822 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 907/1844] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:16.585734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96d527-a577-4881-82f6-d5592ff99541 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:16.588250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cc96d527-a577-4881-82f6-d5592ff99541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:16.625689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc96d527-a577-4881-82f6-d5592ff99541 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:16.635720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:16.636783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:16.637133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:16.637444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] PATH_INFO: `/v3/projects/15ea79d7c6114867aeb2a6ffdba2ff44` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:16.639161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15ea79d7c6114867aeb2a6ffdba2ff44)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:16.641156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:16.786541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15ea79d7c6114867aeb2a6ffdba2ff44', 'request_id': 'req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:16.809485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15ea79d7c6114867aeb2a6ffdba2ff44', 'request_id': 'req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:16.828349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15ea79d7c6114867aeb2a6ffdba2ff44', 'request_id': 'req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:16.866429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] The token cache is being invalidate because project 15ea79d7c6114867aeb2a6ffdba2ff44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:16.866744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:16.868516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:16.875298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 938/1845] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:16 2022] DELETE /identity/v3/projects/15ea79d7c6114867aeb2a6ffdba2ff44 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:16.877583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5f5e3-5794-4be3-9ec3-bd10e4856a39 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:16.909612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea5f5e3-5794-4be3-9ec3-bd10e4856a39 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:17.127118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:17.128074 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:17.128377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:17.128663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] PATH_INFO: `/v3/users/bcf75a8843894daeb1a920e25fdc8ccb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:17.132159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf75a8843894daeb1a920e25fdc8ccb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:17.138066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:17.282680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf75a8843894daeb1a920e25fdc8ccb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:17.296257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf75a8843894daeb1a920e25fdc8ccb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:17.298340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf75a8843894daeb1a920e25fdc8ccb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:17.299433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf75a8843894daeb1a920e25fdc8ccb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:17.315418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ce4be29c-91c8-4cb3-b33b-b3cc7477fa1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf75a8843894daeb1a920e25fdc8ccb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:17.356763 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 908/1846] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:16 2022] DELETE /identity/v3/users/bcf75a8843894daeb1a920e25fdc8ccb => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:18.459799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:18.462579 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:18.487029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c12ebe4-ba74-49de-be9f-3ad4dbc2d7a2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:18.694339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:18.799697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:18.803966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:18.804324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:18.804607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] PATH_INFO: `/v3/projects/8f6b5517c1944e47b1a04105676c3503` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:18.822681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6b5517c1944e47b1a04105676c3503)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:18.824784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:18.971551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6b5517c1944e47b1a04105676c3503', 'request_id': 'req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:18.986895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6b5517c1944e47b1a04105676c3503', 'request_id': 'req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:18.995830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6b5517c1944e47b1a04105676c3503', 'request_id': 'req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:19.013594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] The token cache is being invalidate because project 8f6b5517c1944e47b1a04105676c3503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:19.014312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:19.015353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d8a53b3-e23c-4b5e-978e-45e3fc29fdd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:19.017284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 939/1847] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:18 2022] DELETE /identity/v3/projects/8f6b5517c1944e47b1a04105676c3503 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:19.022613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3eda3-97b6-4101-9bc7-d7d4bbf764d7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:19.023639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3eda3-97b6-4101-9bc7-d7d4bbf764d7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:19.024121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3eda3-97b6-4101-9bc7-d7d4bbf764d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:19.057956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0ea3eda3-97b6-4101-9bc7-d7d4bbf764d7 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:19.059636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ea3eda3-97b6-4101-9bc7-d7d4bbf764d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:19.179492 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:19.223367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 909/1848] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:19.231497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66c2ed-ca05-46f7-a03c-862a20521198 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:19.231926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66c2ed-ca05-46f7-a03c-862a20521198 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:19.232335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66c2ed-ca05-46f7-a03c-862a20521198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:19.264758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9e66c2ed-ca05-46f7-a03c-862a20521198 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:19.267641 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9e66c2ed-ca05-46f7-a03c-862a20521198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:19.284864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 940/1849] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:19.287023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3eda3-97b6-4101-9bc7-d7d4bbf764d7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:19.300023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3eda3-97b6-4101-9bc7-d7d4bbf764d7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:19.305564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722f86-ccd3-4594-a6e6-40eda8fd2f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:19.306954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722f86-ccd3-4594-a6e6-40eda8fd2f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:19.307247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722f86-ccd3-4594-a6e6-40eda8fd2f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:19.307571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722f86-ccd3-4594-a6e6-40eda8fd2f1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:19.308276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8722f86-ccd3-4594-a6e6-40eda8fd2f1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:09:19.308582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8722f86-ccd3-4594-a6e6-40eda8fd2f1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:19.310464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8722f86-ccd3-4594-a6e6-40eda8fd2f1b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:19.331099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 910/1850] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:09:19.336771 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66c2ed-ca05-46f7-a03c-862a20521198 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:19.347700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66c2ed-ca05-46f7-a03c-862a20521198 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:19.353305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f16c5c-0b14-46a8-9450-69e3b7121f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:19.354033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f16c5c-0b14-46a8-9450-69e3b7121f07 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:19.354396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f16c5c-0b14-46a8-9450-69e3b7121f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:19.354672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f16c5c-0b14-46a8-9450-69e3b7121f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:19.355538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f16c5c-0b14-46a8-9450-69e3b7121f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:19.357790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f16c5c-0b14-46a8-9450-69e3b7121f07 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:19.398234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 941/1851] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:19 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:19.400219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722f86-ccd3-4594-a6e6-40eda8fd2f1b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:19.420428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722f86-ccd3-4594-a6e6-40eda8fd2f1b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:19.540130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b308c-ff93-4319-ab55-fd21c044cb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:19.541058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b308c-ff93-4319-ab55-fd21c044cb10 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:19.541295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b308c-ff93-4319-ab55-fd21c044cb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:19.541573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380b308c-ff93-4319-ab55-fd21c044cb10 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:19.542250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b308c-ff93-4319-ab55-fd21c044cb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:19.547188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380b308c-ff93-4319-ab55-fd21c044cb10 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:19.577810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 911/1852] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:19 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:19.582247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f16c5c-0b14-46a8-9450-69e3b7121f07 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:19.603641 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f16c5c-0b14-46a8-9450-69e3b7121f07 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:19.611206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b75b6-c681-4835-bfea-c2d9a76f482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:19.612703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b75b6-c681-4835-bfea-c2d9a76f482c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:19.613357 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b75b6-c681-4835-bfea-c2d9a76f482c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:19.614175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89b75b6-c681-4835-bfea-c2d9a76f482c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:19.615081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b75b6-c681-4835-bfea-c2d9a76f482c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:19.617489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89b75b6-c681-4835-bfea-c2d9a76f482c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:19.637959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 942/1853] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:19.641348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b308c-ff93-4319-ab55-fd21c044cb10 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:19.657222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380b308c-ff93-4319-ab55-fd21c044cb10 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:19.666846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597fa41a-0a75-4bd0-bfae-98a84b22fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:19.667526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fa41a-0a75-4bd0-bfae-98a84b22fc78 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:19.667810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fa41a-0a75-4bd0-bfae-98a84b22fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:19.668091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597fa41a-0a75-4bd0-bfae-98a84b22fc78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:19.670143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fa41a-0a75-4bd0-bfae-98a84b22fc78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:19.671812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597fa41a-0a75-4bd0-bfae-98a84b22fc78 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:19.693827 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 912/1854] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:19.697592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b75b6-c681-4835-bfea-c2d9a76f482c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:19.713505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89b75b6-c681-4835-bfea-c2d9a76f482c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:19.728258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b31fc9-d02d-44ef-868c-5e03af12ead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:19.728946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b31fc9-d02d-44ef-868c-5e03af12ead9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:19.729531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b31fc9-d02d-44ef-868c-5e03af12ead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:19.730055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b31fc9-d02d-44ef-868c-5e03af12ead9 admin admin] PATH_INFO: `/v3/projects/23c1c94464e34945928edb6d3c288cde/users/849bad7d15964def97e09cb2b6e885ff/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:19.766858 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b31fc9-d02d-44ef-868c-5e03af12ead9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23c1c94464e34945928edb6d3c288cde, user_id=849bad7d15964def97e09cb2b6e885ff, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:19.773459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b31fc9-d02d-44ef-868c-5e03af12ead9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:19.804605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 943/1855] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:19 2022] PUT /identity/v3/projects/23c1c94464e34945928edb6d3c288cde/users/849bad7d15964def97e09cb2b6e885ff/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:19.816509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6cb2-46f3-42f4-9f39-8c4cce5190c2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:19.817018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6cb2-46f3-42f4-9f39-8c4cce5190c2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:19.817484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6cb2-46f3-42f4-9f39-8c4cce5190c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:19.869388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d3ba6cb2-46f3-42f4-9f39-8c4cce5190c2 None None] MFA Rules not processed for user `849bad7d15964def97e09cb2b6e885ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:19.870717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d3ba6cb2-46f3-42f4-9f39-8c4cce5190c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:20.021999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 913/1856] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:20.026245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b31fc9-d02d-44ef-868c-5e03af12ead9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:20.144123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b31fc9-d02d-44ef-868c-5e03af12ead9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:20.161551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447eb94-5337-4694-99d7-bcda02de93b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:20.161551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447eb94-5337-4694-99d7-bcda02de93b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:20.162985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447eb94-5337-4694-99d7-bcda02de93b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:20.163369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447eb94-5337-4694-99d7-bcda02de93b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:20.164079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447eb94-5337-4694-99d7-bcda02de93b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:20.166135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447eb94-5337-4694-99d7-bcda02de93b4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:20.187708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:20.237380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 944/1857] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:20 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:20.239683 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6cb2-46f3-42f4-9f39-8c4cce5190c2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:20.254646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6cb2-46f3-42f4-9f39-8c4cce5190c2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:20.308947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447eb94-5337-4694-99d7-bcda02de93b4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:20.310966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1447eb94-5337-4694-99d7-bcda02de93b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:20.336295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:20.382735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a826af-8d6a-4559-9f21-8bf5c13bad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:20.383254 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a826af-8d6a-4559-9f21-8bf5c13bad9a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:20.383593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a826af-8d6a-4559-9f21-8bf5c13bad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:20.383874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a826af-8d6a-4559-9f21-8bf5c13bad9a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:20.384545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a826af-8d6a-4559-9f21-8bf5c13bad9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:20.388991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a826af-8d6a-4559-9f21-8bf5c13bad9a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:20.456241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 914/1858] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:20 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:20.494962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a826af-8d6a-4559-9f21-8bf5c13bad9a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:20.543276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a826af-8d6a-4559-9f21-8bf5c13bad9a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:20.555100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447eb94-5337-4694-99d7-bcda02de93b4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:20.560958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106d561-9e2e-480d-82fa-4f6bf2f70400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:20.563256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d561-9e2e-480d-82fa-4f6bf2f70400 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:20.563893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d561-9e2e-480d-82fa-4f6bf2f70400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:20.566085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106d561-9e2e-480d-82fa-4f6bf2f70400 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:20.567092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106d561-9e2e-480d-82fa-4f6bf2f70400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:20.569905 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106d561-9e2e-480d-82fa-4f6bf2f70400 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:20.570957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:20.572649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:20.573419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:20.573975 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] PATH_INFO: `/v3/users/a6548b2c559a4db5ad01386db2bb9080` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:20.578180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6548b2c559a4db5ad01386db2bb9080)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:20.582293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:20.615862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 915/1859] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:20.615862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106d561-9e2e-480d-82fa-4f6bf2f70400 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:20.632046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106d561-9e2e-480d-82fa-4f6bf2f70400 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:20.637789 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ab86c-394c-4e72-85a4-e229d9f407db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:20.638783 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ab86c-394c-4e72-85a4-e229d9f407db admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:20.639151 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ab86c-394c-4e72-85a4-e229d9f407db admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:20.639633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ab86c-394c-4e72-85a4-e229d9f407db admin admin] PATH_INFO: `/v3/projects/75c77383059b4736a3f1a3bcc69b98ff/users/671f43b77e434e6d87658e3e623b6198/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:20.660628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ab86c-394c-4e72-85a4-e229d9f407db admin admin] RBAC: Authorizing `identity:create_grant(project_id=75c77383059b4736a3f1a3bcc69b98ff, user_id=671f43b77e434e6d87658e3e623b6198, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:20.664838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ab86c-394c-4e72-85a4-e229d9f407db admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:20.685225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 916/1860] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:20 2022] PUT /identity/v3/projects/75c77383059b4736a3f1a3bcc69b98ff/users/671f43b77e434e6d87658e3e623b6198/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:20.689643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ab86c-394c-4e72-85a4-e229d9f407db admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:20.738523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6548b2c559a4db5ad01386db2bb9080'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:20.739660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6548b2c559a4db5ad01386db2bb9080'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:20.742219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6548b2c559a4db5ad01386db2bb9080'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:20.770899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6548b2c559a4db5ad01386db2bb9080'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:20.800695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5e4262c3-a100-4efb-a4a4-3f5f68e1d44b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6548b2c559a4db5ad01386db2bb9080'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:20.814252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ab86c-394c-4e72-85a4-e229d9f407db admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:20.822754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c111df-b4da-4e60-bfbe-fd4af53b21ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:20.824574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c111df-b4da-4e60-bfbe-fd4af53b21ba admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:20.825047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c111df-b4da-4e60-bfbe-fd4af53b21ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:20.825406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c111df-b4da-4e60-bfbe-fd4af53b21ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:20.826640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c111df-b4da-4e60-bfbe-fd4af53b21ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:20.828541 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c111df-b4da-4e60-bfbe-fd4af53b21ba admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:20.835811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 945/1861] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:20 2022] DELETE /identity/v3/users/a6548b2c559a4db5ad01386db2bb9080 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:20.842670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993c885-3f0c-4d56-bc1b-41b2ddaa683a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:20.842964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993c885-3f0c-4d56-bc1b-41b2ddaa683a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:20.843253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993c885-3f0c-4d56-bc1b-41b2ddaa683a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:20.845954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 917/1862] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:09:20.848191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c111df-b4da-4e60-bfbe-fd4af53b21ba admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:20.850916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f7c111df-b4da-4e60-bfbe-fd4af53b21ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:20.868910 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c111df-b4da-4e60-bfbe-fd4af53b21ba admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:20.869173 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a993c885-3f0c-4d56-bc1b-41b2ddaa683a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:20.870520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a993c885-3f0c-4d56-bc1b-41b2ddaa683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:20.981051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 946/1863] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:20.982077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0e8cd-5585-4e3d-bdce-d1cefc77e0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:20.983628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0e8cd-5585-4e3d-bdce-d1cefc77e0b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:20.986455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0e8cd-5585-4e3d-bdce-d1cefc77e0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:20.986753 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0e8cd-5585-4e3d-bdce-d1cefc77e0b3 admin admin] PATH_INFO: `/v3/projects/75c77383059b4736a3f1a3bcc69b98ff/users/671f43b77e434e6d87658e3e623b6198/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:20.989357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0e8cd-5585-4e3d-bdce-d1cefc77e0b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75c77383059b4736a3f1a3bcc69b98ff, user_id=671f43b77e434e6d87658e3e623b6198, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:20.993244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0e8cd-5585-4e3d-bdce-d1cefc77e0b3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:20.996615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993c885-3f0c-4d56-bc1b-41b2ddaa683a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:20.998743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a993c885-3f0c-4d56-bc1b-41b2ddaa683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:21.020713 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 918/1864] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:20 2022] PUT /identity/v3/projects/75c77383059b4736a3f1a3bcc69b98ff/users/671f43b77e434e6d87658e3e623b6198/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:21.029569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5810d2-f5bb-49c1-a567-7adf017cae56 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:21.029569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5810d2-f5bb-49c1-a567-7adf017cae56 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:21.029569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5810d2-f5bb-49c1-a567-7adf017cae56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:21.031034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:21.072568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5d5810d2-f5bb-49c1-a567-7adf017cae56 None None] MFA Rules not processed for user `671f43b77e434e6d87658e3e623b6198`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:21.074440 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5d5810d2-f5bb-49c1-a567-7adf017cae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:21.101110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993c885-3f0c-4d56-bc1b-41b2ddaa683a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:21.103730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:21.190893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 919/1865] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:21.202757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87f000-7d0c-4c09-9b93-aa37b5f7f180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:21.203712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87f000-7d0c-4c09-9b93-aa37b5f7f180 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:21.204001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87f000-7d0c-4c09-9b93-aa37b5f7f180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:21.204277 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87f000-7d0c-4c09-9b93-aa37b5f7f180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:21.204534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5810d2-f5bb-49c1-a567-7adf017cae56 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:21.206912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5d5810d2-f5bb-49c1-a567-7adf017cae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:21.299837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87f000-7d0c-4c09-9b93-aa37b5f7f180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:21.302013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87f000-7d0c-4c09-9b93-aa37b5f7f180 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:21.313241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5810d2-f5bb-49c1-a567-7adf017cae56 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:21.314350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 947/1866] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:09:21.320745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018815fb-19dc-4df4-8fb5-b79462c37970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:21.323001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018815fb-19dc-4df4-8fb5-b79462c37970 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:21.323625 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018815fb-19dc-4df4-8fb5-b79462c37970 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:21.324223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018815fb-19dc-4df4-8fb5-b79462c37970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:21.342535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018815fb-19dc-4df4-8fb5-b79462c37970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:21.344859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018815fb-19dc-4df4-8fb5-b79462c37970 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:21.365623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 920/1867] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:21.406409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87f000-7d0c-4c09-9b93-aa37b5f7f180 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:21.431143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87f000-7d0c-4c09-9b93-aa37b5f7f180 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:21.436597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:21.437280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:21.437720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:21.438130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] PATH_INFO: `/v3/users/849bad7d15964def97e09cb2b6e885ff` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:21.439570 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=849bad7d15964def97e09cb2b6e885ff)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:21.441721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:21.548564 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '849bad7d15964def97e09cb2b6e885ff'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:21.549376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '849bad7d15964def97e09cb2b6e885ff'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:21.550115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '849bad7d15964def97e09cb2b6e885ff'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:21.570137 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '849bad7d15964def97e09cb2b6e885ff'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:21.598839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '849bad7d15964def97e09cb2b6e885ff'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:21.608310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:21.629109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e5ccd-9dcd-437d-ae79-dd9e5648f17c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:21.629870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e5ccd-9dcd-437d-ae79-dd9e5648f17c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:21.630364 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e5ccd-9dcd-437d-ae79-dd9e5648f17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:21.634791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 948/1868] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:21 2022] DELETE /identity/v3/users/849bad7d15964def97e09cb2b6e885ff => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:21.661090 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-043e5ccd-9dcd-437d-ae79-dd9e5648f17c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:21.663357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-043e5ccd-9dcd-437d-ae79-dd9e5648f17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:21.801182 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 921/1869] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:21.803428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:21.807928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:21.816757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e5ccd-9dcd-437d-ae79-dd9e5648f17c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:21.819257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-043e5ccd-9dcd-437d-ae79-dd9e5648f17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:21.838034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4ef9a-7cd5-453a-9cff-0dd77fb39702 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:21.850687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:21.851690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:21.852447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:21.852730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] PATH_INFO: `/v3/projects/86c3095b2979482c935bd3f32c5d7ec3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:21.855714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=86c3095b2979482c935bd3f32c5d7ec3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:21.860267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:21.929708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e5ccd-9dcd-437d-ae79-dd9e5648f17c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:21.960747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:21.982506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86c3095b2979482c935bd3f32c5d7ec3', 'request_id': 'req-74406307-4524-4b7c-b061-c386ec6a76ca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:22.011061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86c3095b2979482c935bd3f32c5d7ec3', 'request_id': 'req-74406307-4524-4b7c-b061-c386ec6a76ca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:22.031703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86c3095b2979482c935bd3f32c5d7ec3', 'request_id': 'req-74406307-4524-4b7c-b061-c386ec6a76ca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:22.061277 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] The token cache is being invalidate because project 86c3095b2979482c935bd3f32c5d7ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:22.061735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:22.062634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:22.069958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 949/1870] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:21 2022] DELETE /identity/v3/projects/86c3095b2979482c935bd3f32c5d7ec3 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:22.071775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:22.074015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:22.079897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5cd61-32b4-4afd-a2e5-f0888bf6201d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:22.083339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5cd61-32b4-4afd-a2e5-f0888bf6201d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:22.084252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5cd61-32b4-4afd-a2e5-f0888bf6201d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:22.085266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b5cd61-32b4-4afd-a2e5-f0888bf6201d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:22.097382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-48b5cd61-32b4-4afd-a2e5-f0888bf6201d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:22.207635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5cd61-32b4-4afd-a2e5-f0888bf6201d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:22.209197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b5cd61-32b4-4afd-a2e5-f0888bf6201d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:22.225226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 922/1871] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:22.253481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74406307-4524-4b7c-b061-c386ec6a76ca admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:22.265019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:22.267138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:22.268056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:22.268393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] PATH_INFO: `/v3/users/2795e9eb58f84df680174877884acd7a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:22.271824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=2795e9eb58f84df680174877884acd7a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:22.276626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:22.355240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2795e9eb58f84df680174877884acd7a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:22.356461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2795e9eb58f84df680174877884acd7a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:22.357130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2795e9eb58f84df680174877884acd7a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:22.372687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2795e9eb58f84df680174877884acd7a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:22.401080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2795e9eb58f84df680174877884acd7a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:22.416772 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:22.449508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 950/1872] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:22 2022] DELETE /identity/v3/users/2795e9eb58f84df680174877884acd7a => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:22.986706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5cd61-32b4-4afd-a2e5-f0888bf6201d service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:23.001171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b5cd61-32b4-4afd-a2e5-f0888bf6201d service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:23.022263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:23.026488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:23.168647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:23.169277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:23.170428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:23.170681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] PATH_INFO: `/v3/projects/38b56d4fd326438c9cff1414f88f93c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:23.172994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b56d4fd326438c9cff1414f88f93c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:23.176896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:23.215905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a4f801-9b6b-439a-98d0-8196abc3f5ed admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:23.224747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:23.225941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:23.226304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:23.227361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] PATH_INFO: `/v3/projects/23c1c94464e34945928edb6d3c288cde` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:23.229448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] RBAC: Authorizing `identity:delete_project(project_id=23c1c94464e34945928edb6d3c288cde)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:23.231833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:23.264391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b56d4fd326438c9cff1414f88f93c5', 'request_id': 'req-f0b01d25-04f3-4fda-9e60-3afced8968e5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.277512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b56d4fd326438c9cff1414f88f93c5', 'request_id': 'req-f0b01d25-04f3-4fda-9e60-3afced8968e5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.289601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b56d4fd326438c9cff1414f88f93c5', 'request_id': 'req-f0b01d25-04f3-4fda-9e60-3afced8968e5'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.294215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23c1c94464e34945928edb6d3c288cde', 'request_id': 'req-9966d7b2-9025-4873-a6bb-5cdb391ef06a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.295510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:23.321560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23c1c94464e34945928edb6d3c288cde', 'request_id': 'req-9966d7b2-9025-4873-a6bb-5cdb391ef06a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.325547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] The token cache is being invalidate because project 38b56d4fd326438c9cff1414f88f93c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:23.326201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.327199 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f0b01d25-04f3-4fda-9e60-3afced8968e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.329142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 923/1873] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:22 2022] DELETE /identity/v3/projects/38b56d4fd326438c9cff1414f88f93c5 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:23.336434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ad8f4-f0d9-4a07-b05c-429c0e7aa1c2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:23.336719 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ad8f4-f0d9-4a07-b05c-429c0e7aa1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:23.336990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ad8f4-f0d9-4a07-b05c-429c0e7aa1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:23.342130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23c1c94464e34945928edb6d3c288cde', 'request_id': 'req-9966d7b2-9025-4873-a6bb-5cdb391ef06a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.360127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9c4ad8f4-f0d9-4a07-b05c-429c0e7aa1c2 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:23.360448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9c4ad8f4-f0d9-4a07-b05c-429c0e7aa1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:23.368898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] The token cache is being invalidate because project 23c1c94464e34945928edb6d3c288cde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:23.369442 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.370726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.373537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 951/1874] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:23 2022] DELETE /identity/v3/projects/23c1c94464e34945928edb6d3c288cde => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:23.378864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:23.381538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:23.476453 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d7b2-9025-4873-a6bb-5cdb391ef06a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:23.477042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 924/1875] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:23.483027 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:23.483688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:23.483971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:23.484245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] PATH_INFO: `/v3/users/671f43b77e434e6d87658e3e623b6198` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:23.485483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=671f43b77e434e6d87658e3e623b6198)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:23.487657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:23.490804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55bcc4-290f-4ded-8199-baf3ae1f9e88 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:23.491421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55bcc4-290f-4ded-8199-baf3ae1f9e88 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:23.491736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55bcc4-290f-4ded-8199-baf3ae1f9e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:23.519419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5f55bcc4-290f-4ded-8199-baf3ae1f9e88 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:23.521425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5f55bcc4-290f-4ded-8199-baf3ae1f9e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:23.523474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:23.536941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 925/1876] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:23.542964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55bcc4-290f-4ded-8199-baf3ae1f9e88 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:23.551347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '671f43b77e434e6d87658e3e623b6198'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.552076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '671f43b77e434e6d87658e3e623b6198'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.553386 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '671f43b77e434e6d87658e3e623b6198'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.563670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '671f43b77e434e6d87658e3e623b6198'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.585246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '671f43b77e434e6d87658e3e623b6198'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:23.591825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f55bcc4-290f-4ded-8199-baf3ae1f9e88 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:23.605983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4783b2b-2ef4-4ef1-9d16-15eb7339b3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:23.606531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 952/1877] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:23 2022] DELETE /identity/v3/users/671f43b77e434e6d87658e3e623b6198 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:23.607676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783b2b-2ef4-4ef1-9d16-15eb7339b3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:23.608176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783b2b-2ef4-4ef1-9d16-15eb7339b3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:23.608685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4783b2b-2ef4-4ef1-9d16-15eb7339b3c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:23.610032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4783b2b-2ef4-4ef1-9d16-15eb7339b3c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:09:23.610602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4783b2b-2ef4-4ef1-9d16-15eb7339b3c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:23.613680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4783b2b-2ef4-4ef1-9d16-15eb7339b3c7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:23.647872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 926/1878] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:09:23.648169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:23.649840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:23.845988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02614-d427-4143-a670-33442f9fa3f0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:23.858200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83db4ee-6188-48cc-9141-e45c9d862469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:23.858902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83db4ee-6188-48cc-9141-e45c9d862469 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:23.859184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83db4ee-6188-48cc-9141-e45c9d862469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:23.859459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83db4ee-6188-48cc-9141-e45c9d862469 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:23.864165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83db4ee-6188-48cc-9141-e45c9d862469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:23.870497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83db4ee-6188-48cc-9141-e45c9d862469 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:24.006429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 953/1879] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:23 2022] POST /identity/v3/projects => generated 365 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:24.013425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4783b2b-2ef4-4ef1-9d16-15eb7339b3c7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:24.039154 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4783b2b-2ef4-4ef1-9d16-15eb7339b3c7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:24.213596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd527a-6fee-4579-add4-eae79d0e2cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:24.214700 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd527a-6fee-4579-add4-eae79d0e2cf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:24.215397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd527a-6fee-4579-add4-eae79d0e2cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:24.215397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd527a-6fee-4579-add4-eae79d0e2cf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:24.216311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd527a-6fee-4579-add4-eae79d0e2cf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:24.219120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd527a-6fee-4579-add4-eae79d0e2cf9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:24.270955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83db4ee-6188-48cc-9141-e45c9d862469 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:24.273555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 927/1880] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:24 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:24.306237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83db4ee-6188-48cc-9141-e45c9d862469 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:24.314620 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ff3f7-214b-488a-b097-74fc744ad5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:24.315368 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ff3f7-214b-488a-b097-74fc744ad5aa admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:24.315681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ff3f7-214b-488a-b097-74fc744ad5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:24.315991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ff3f7-214b-488a-b097-74fc744ad5aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:24.316880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ff3f7-214b-488a-b097-74fc744ad5aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:24.321159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ff3f7-214b-488a-b097-74fc744ad5aa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:24.338072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 954/1881] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:24.342452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd527a-6fee-4579-add4-eae79d0e2cf9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:24.371153 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd527a-6fee-4579-add4-eae79d0e2cf9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:24.402828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b7a1f-389b-476f-b9bb-2e951cadc06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:24.404537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b7a1f-389b-476f-b9bb-2e951cadc06f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:24.405120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b7a1f-389b-476f-b9bb-2e951cadc06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:24.405869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b7a1f-389b-476f-b9bb-2e951cadc06f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:24.407397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b7a1f-389b-476f-b9bb-2e951cadc06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:24.409686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b7a1f-389b-476f-b9bb-2e951cadc06f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:24.439647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 928/1882] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:24.441297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ff3f7-214b-488a-b097-74fc744ad5aa admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:24.454374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ff3f7-214b-488a-b097-74fc744ad5aa admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:24.460658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf3054-e365-4829-916d-9c5ee4ea4224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:24.461630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3054-e365-4829-916d-9c5ee4ea4224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:24.462075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3054-e365-4829-916d-9c5ee4ea4224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:24.462354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3054-e365-4829-916d-9c5ee4ea4224 admin admin] PATH_INFO: `/v3/projects/371ee764c7a84ac09c4217a3eeffe2a8/users/371e98756d4e4623a6c846279a0db0a2/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:24.484680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf3054-e365-4829-916d-9c5ee4ea4224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=371ee764c7a84ac09c4217a3eeffe2a8, user_id=371e98756d4e4623a6c846279a0db0a2, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:24.487791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf3054-e365-4829-916d-9c5ee4ea4224 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:24.510411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 955/1883] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:24 2022] PUT /identity/v3/projects/371ee764c7a84ac09c4217a3eeffe2a8/users/371e98756d4e4623a6c846279a0db0a2/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:24.526457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c92bf9-b0a3-476b-897f-c85b5e2ff769 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:24.527520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c92bf9-b0a3-476b-897f-c85b5e2ff769 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:24.528239 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c92bf9-b0a3-476b-897f-c85b5e2ff769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:24.578939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e8c92bf9-b0a3-476b-897f-c85b5e2ff769 None None] MFA Rules not processed for user `371e98756d4e4623a6c846279a0db0a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:24.580524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e8c92bf9-b0a3-476b-897f-c85b5e2ff769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:24.635655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:24.693237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf3054-e365-4829-916d-9c5ee4ea4224 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:24.695541 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 929/1884] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:24.695687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-05cf3054-e365-4829-916d-9c5ee4ea4224 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:24.701750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c92bf9-b0a3-476b-897f-c85b5e2ff769 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:24.813143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c92bf9-b0a3-476b-897f-c85b5e2ff769 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:24.830830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4bd57-c08e-45c9-8de6-168b4744185d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:24.832144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4bd57-c08e-45c9-8de6-168b4744185d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:24.834888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4bd57-c08e-45c9-8de6-168b4744185d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:24.835518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4bd57-c08e-45c9-8de6-168b4744185d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:24.836379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4bd57-c08e-45c9-8de6-168b4744185d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:24.842594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4bd57-c08e-45c9-8de6-168b4744185d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:24.923235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:24.923885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 930/1885] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:24 2022] POST /identity/v3/projects => generated 367 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:24.924984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4bd57-c08e-45c9-8de6-168b4744185d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:24.939066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf3054-e365-4829-916d-9c5ee4ea4224 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.081013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4bd57-c08e-45c9-8de6-168b4744185d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.089253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.090666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.091336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.093654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] PATH_INFO: `/v3/projects/75c77383059b4736a3f1a3bcc69b98ff` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.094488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9546cca-ecaf-4684-a266-c7dc45e3c671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.095213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546cca-ecaf-4684-a266-c7dc45e3c671 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.095602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546cca-ecaf-4684-a266-c7dc45e3c671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.095953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9546cca-ecaf-4684-a266-c7dc45e3c671 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.096680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9546cca-ecaf-4684-a266-c7dc45e3c671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.096972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=75c77383059b4736a3f1a3bcc69b98ff)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.099335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9546cca-ecaf-4684-a266-c7dc45e3c671 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:25.101333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:25.135555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 931/1886] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:24 2022] POST /identity/v3/users => generated 290 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:25.139247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9546cca-ecaf-4684-a266-c7dc45e3c671 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:25.159011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9546cca-ecaf-4684-a266-c7dc45e3c671 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.170684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77355a-784c-4b14-9871-e6e7051b5f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.171733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77355a-784c-4b14-9871-e6e7051b5f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.172755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77355a-784c-4b14-9871-e6e7051b5f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.173186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77355a-784c-4b14-9871-e6e7051b5f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.174677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77355a-784c-4b14-9871-e6e7051b5f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.176806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f77355a-784c-4b14-9871-e6e7051b5f1d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:25.215639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 932/1887] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:09:25.220809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77355a-784c-4b14-9871-e6e7051b5f1d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:25.232665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77355a-784c-4b14-9871-e6e7051b5f1d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.251319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75c77383059b4736a3f1a3bcc69b98ff', 'request_id': 'req-a039b4f5-769b-4e54-8227-7b7d467beb6a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:25.266043 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75c77383059b4736a3f1a3bcc69b98ff', 'request_id': 'req-a039b4f5-769b-4e54-8227-7b7d467beb6a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:25.283430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75c77383059b4736a3f1a3bcc69b98ff', 'request_id': 'req-a039b4f5-769b-4e54-8227-7b7d467beb6a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:25.289656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:25.313838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] The token cache is being invalidate because project 75c77383059b4736a3f1a3bcc69b98ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:25.313965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:25.314795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a039b4f5-769b-4e54-8227-7b7d467beb6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:25.315026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d79a55-db39-48b2-b2da-2c09dc161a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.316153 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79a55-db39-48b2-b2da-2c09dc161a66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.316435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79a55-db39-48b2-b2da-2c09dc161a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.316772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d79a55-db39-48b2-b2da-2c09dc161a66 admin admin] PATH_INFO: `/v3/projects/9ebf6097b7f741b5830c749db9bd40a4/users/a5a853d07b014523b988719632110b23/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.321862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 956/1888] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:24 2022] DELETE /identity/v3/projects/75c77383059b4736a3f1a3bcc69b98ff => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:25.324961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d347b8a-663d-4331-bcf0-75d6dda7ca6d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.325189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d347b8a-663d-4331-bcf0-75d6dda7ca6d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.325731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d347b8a-663d-4331-bcf0-75d6dda7ca6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.361454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d79a55-db39-48b2-b2da-2c09dc161a66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebf6097b7f741b5830c749db9bd40a4, user_id=a5a853d07b014523b988719632110b23, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.369857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d79a55-db39-48b2-b2da-2c09dc161a66 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:25.386532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 933/1889] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:25 2022] PUT /identity/v3/projects/9ebf6097b7f741b5830c749db9bd40a4/users/a5a853d07b014523b988719632110b23/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:25.388096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7d347b8a-663d-4331-bcf0-75d6dda7ca6d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:25.390769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d79a55-db39-48b2-b2da-2c09dc161a66 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:25.390919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7d347b8a-663d-4331-bcf0-75d6dda7ca6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:25.393806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e5d79a55-db39-48b2-b2da-2c09dc161a66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:25.481421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d79a55-db39-48b2-b2da-2c09dc161a66 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.487509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c156d-1f66-448e-874b-61f4ecfadff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.488266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c156d-1f66-448e-874b-61f4ecfadff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.488623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c156d-1f66-448e-874b-61f4ecfadff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.488997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c156d-1f66-448e-874b-61f4ecfadff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.490429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c156d-1f66-448e-874b-61f4ecfadff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.492221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c156d-1f66-448e-874b-61f4ecfadff0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:25.499643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 957/1890] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:25.508890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 934/1891] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:09:25.510812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c156d-1f66-448e-874b-61f4ecfadff0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:25.511000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22894496-6ea2-44c2-8e92-fcb31e337f69 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.511457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22894496-6ea2-44c2-8e92-fcb31e337f69 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.511703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22894496-6ea2-44c2-8e92-fcb31e337f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.526488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c156d-1f66-448e-874b-61f4ecfadff0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.533197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62836248-0891-4ae1-a949-54c80220523f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.533197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62836248-0891-4ae1-a949-54c80220523f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.534031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62836248-0891-4ae1-a949-54c80220523f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.534441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62836248-0891-4ae1-a949-54c80220523f admin admin] PATH_INFO: `/v3/projects/9ebf6097b7f741b5830c749db9bd40a4/users/a5a853d07b014523b988719632110b23/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.535992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-22894496-6ea2-44c2-8e92-fcb31e337f69 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:25.537479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-22894496-6ea2-44c2-8e92-fcb31e337f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:25.538950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62836248-0891-4ae1-a949-54c80220523f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ebf6097b7f741b5830c749db9bd40a4, user_id=a5a853d07b014523b988719632110b23, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.541897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62836248-0891-4ae1-a949-54c80220523f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:25.548572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 958/1892] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:25.553708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22894496-6ea2-44c2-8e92-fcb31e337f69 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:25.562621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 935/1893] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:25 2022] PUT /identity/v3/projects/9ebf6097b7f741b5830c749db9bd40a4/users/a5a853d07b014523b988719632110b23/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:25.570847 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd867cc7-3f68-4f0d-a5c7-b5927d185264 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.571136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd867cc7-3f68-4f0d-a5c7-b5927d185264 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.571416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd867cc7-3f68-4f0d-a5c7-b5927d185264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.571843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22894496-6ea2-44c2-8e92-fcb31e337f69 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.607216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bd867cc7-3f68-4f0d-a5c7-b5927d185264 None None] MFA Rules not processed for user `a5a853d07b014523b988719632110b23`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:25.609049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd867cc7-3f68-4f0d-a5c7-b5927d185264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:25.610897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:25.653677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5e3ae-7284-4834-9358-130c65777c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.654450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5e3ae-7284-4834-9358-130c65777c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.654736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5e3ae-7284-4834-9358-130c65777c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.655035 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5e3ae-7284-4834-9358-130c65777c8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.655907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5e3ae-7284-4834-9358-130c65777c8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:09:25.656218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5e3ae-7284-4834-9358-130c65777c8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.658102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5e3ae-7284-4834-9358-130c65777c8b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:25.674773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 959/1894] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:09:25.681586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5e3ae-7284-4834-9358-130c65777c8b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:25.697602 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5e3ae-7284-4834-9358-130c65777c8b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.705658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbeec7-4eca-40eb-9899-cc723a08fb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.706341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbeec7-4eca-40eb-9899-cc723a08fb47 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.706636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbeec7-4eca-40eb-9899-cc723a08fb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.706906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbeec7-4eca-40eb-9899-cc723a08fb47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.708307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbeec7-4eca-40eb-9899-cc723a08fb47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.710680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbeec7-4eca-40eb-9899-cc723a08fb47 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:25.723788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 936/1895] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:25.737095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd867cc7-3f68-4f0d-a5c7-b5927d185264 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:25.738894 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd867cc7-3f68-4f0d-a5c7-b5927d185264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:25.751612 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 960/1896] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:25 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:25.758687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbeec7-4eca-40eb-9899-cc723a08fb47 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:25.811093 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:25.840997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd867cc7-3f68-4f0d-a5c7-b5927d185264 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.869556 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbeec7-4eca-40eb-9899-cc723a08fb47 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.875218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75902012-de72-4799-acc8-1b982fa9869a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.876192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75902012-de72-4799-acc8-1b982fa9869a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.876622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75902012-de72-4799-acc8-1b982fa9869a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.877072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75902012-de72-4799-acc8-1b982fa9869a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.878160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75902012-de72-4799-acc8-1b982fa9869a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.881029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75902012-de72-4799-acc8-1b982fa9869a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:25.915275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47ef24-8b6c-4265-84b3-453e37946429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.916059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47ef24-8b6c-4265-84b3-453e37946429 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.916453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47ef24-8b6c-4265-84b3-453e37946429 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.916816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47ef24-8b6c-4265-84b3-453e37946429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.917679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 961/1897] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:25 2022] POST /identity/v3/users => generated 286 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:25.921847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75902012-de72-4799-acc8-1b982fa9869a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:25.934824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75902012-de72-4799-acc8-1b982fa9869a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.941099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb77626-e5ce-4b83-a5a3-1ea244d8c67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.941732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb77626-e5ce-4b83-a5a3-1ea244d8c67f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.942091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb77626-e5ce-4b83-a5a3-1ea244d8c67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.942402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb77626-e5ce-4b83-a5a3-1ea244d8c67f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.943670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb77626-e5ce-4b83-a5a3-1ea244d8c67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.945519 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb77626-e5ce-4b83-a5a3-1ea244d8c67f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:25.962447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 962/1898] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:25.964136 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb77626-e5ce-4b83-a5a3-1ea244d8c67f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:25.977302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb77626-e5ce-4b83-a5a3-1ea244d8c67f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:25.984021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae010c3-e0b8-46b4-b3ec-c23a9527ccca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:25.984747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae010c3-e0b8-46b4-b3ec-c23a9527ccca admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:25.985085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae010c3-e0b8-46b4-b3ec-c23a9527ccca admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:25.985449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae010c3-e0b8-46b4-b3ec-c23a9527ccca admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:25.986703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae010c3-e0b8-46b4-b3ec-c23a9527ccca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:25.988439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae010c3-e0b8-46b4-b3ec-c23a9527ccca admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:26.014174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 963/1899] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:09:26.018803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47ef24-8b6c-4265-84b3-453e37946429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:26.020838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47ef24-8b6c-4265-84b3-453e37946429 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:26.022763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae010c3-e0b8-46b4-b3ec-c23a9527ccca admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:26.059223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 937/1900] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:26.059361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae010c3-e0b8-46b4-b3ec-c23a9527ccca admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:26.059361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871c165-cf46-44ed-921d-e3e7db4cb93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:26.059361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871c165-cf46-44ed-921d-e3e7db4cb93f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:26.059361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871c165-cf46-44ed-921d-e3e7db4cb93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:26.059759 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871c165-cf46-44ed-921d-e3e7db4cb93f admin admin] PATH_INFO: `/v3/projects/db251662b2974d7bac68a19a6155a5f0/users/7c6664930d7f45e687976db7c997a022/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:26.080997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871c165-cf46-44ed-921d-e3e7db4cb93f admin admin] RBAC: Authorizing `identity:create_grant(project_id=db251662b2974d7bac68a19a6155a5f0, user_id=7c6664930d7f45e687976db7c997a022, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:26.093813 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871c165-cf46-44ed-921d-e3e7db4cb93f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:26.112571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 964/1901] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:26 2022] PUT /identity/v3/projects/db251662b2974d7bac68a19a6155a5f0/users/7c6664930d7f45e687976db7c997a022/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:26.123847 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a512997-611a-4011-b8b4-2ffbf302673a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:26.125198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a512997-611a-4011-b8b4-2ffbf302673a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:26.125486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a512997-611a-4011-b8b4-2ffbf302673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:26.165215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9a512997-611a-4011-b8b4-2ffbf302673a None None] MFA Rules not processed for user `7c6664930d7f45e687976db7c997a022`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:26.166731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9a512997-611a-4011-b8b4-2ffbf302673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:26.169427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871c165-cf46-44ed-921d-e3e7db4cb93f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:26.247290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:26.256330 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 938/1902] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:26.270635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871c165-cf46-44ed-921d-e3e7db4cb93f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:26.278370 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848fb59-cef8-483f-8fdf-971249115e5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:26.279270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848fb59-cef8-483f-8fdf-971249115e5a service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:26.279657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848fb59-cef8-483f-8fdf-971249115e5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:26.280048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b848fb59-cef8-483f-8fdf-971249115e5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:26.282788 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b848fb59-cef8-483f-8fdf-971249115e5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:26.368898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848fb59-cef8-483f-8fdf-971249115e5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:26.370707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b848fb59-cef8-483f-8fdf-971249115e5a service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:26.393961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 965/1903] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:09:26.435144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a512997-611a-4011-b8b4-2ffbf302673a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:26.438299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9a512997-611a-4011-b8b4-2ffbf302673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:26.471415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a512997-611a-4011-b8b4-2ffbf302673a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:26.479850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884932de-8c12-4e01-9262-c7f209ae5bdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:26.482176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884932de-8c12-4e01-9262-c7f209ae5bdb service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:26.482685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884932de-8c12-4e01-9262-c7f209ae5bdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:26.483136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884932de-8c12-4e01-9262-c7f209ae5bdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:26.513068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884932de-8c12-4e01-9262-c7f209ae5bdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:26.517146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884932de-8c12-4e01-9262-c7f209ae5bdb service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:26.528900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 939/1904] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:09:26.560271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848fb59-cef8-483f-8fdf-971249115e5a service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:26.685284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b848fb59-cef8-483f-8fdf-971249115e5a service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:26.691369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:26.692023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:26.692298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:26.692607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] PATH_INFO: `/v3/users/7c6664930d7f45e687976db7c997a022` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:26.694082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c6664930d7f45e687976db7c997a022)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:26.696103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:26.780687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c6664930d7f45e687976db7c997a022'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:26.782437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c6664930d7f45e687976db7c997a022'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:26.783479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c6664930d7f45e687976db7c997a022'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:26.798890 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c6664930d7f45e687976db7c997a022'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:26.824956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c6664930d7f45e687976db7c997a022'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:26.862224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 966/1905] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:26 2022] DELETE /identity/v3/users/7c6664930d7f45e687976db7c997a022 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:26.869134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80bb1c-0863-4256-b786-9aa54723cbb2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:26.869838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80bb1c-0863-4256-b786-9aa54723cbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:26.870977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80bb1c-0863-4256-b786-9aa54723cbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:26.907952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-be80bb1c-0863-4256-b786-9aa54723cbb2 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:26.909468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-be80bb1c-0863-4256-b786-9aa54723cbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:26.928328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:27.020454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 940/1906] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:27.036257 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:27.039411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:27.072825 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:27.126151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059a2e5-0c5a-4cef-bad1-90d66b8d7786 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:27.131623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f04d53-7cd9-4494-89a8-2ed6b5effe26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:27.132323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f04d53-7cd9-4494-89a8-2ed6b5effe26 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:27.132606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f04d53-7cd9-4494-89a8-2ed6b5effe26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:27.132885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f04d53-7cd9-4494-89a8-2ed6b5effe26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:27.145486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f04d53-7cd9-4494-89a8-2ed6b5effe26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:27.147116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f04d53-7cd9-4494-89a8-2ed6b5effe26 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:27.158251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 967/1907] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:09:27.504113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80bb1c-0863-4256-b786-9aa54723cbb2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:27.506322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-be80bb1c-0863-4256-b786-9aa54723cbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:27.525836 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80bb1c-0863-4256-b786-9aa54723cbb2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:27.531561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:27.532345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:27.532479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:27.533015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] PATH_INFO: `/v3/projects/db251662b2974d7bac68a19a6155a5f0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:27.534429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db251662b2974d7bac68a19a6155a5f0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:27.539862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:27.595520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db251662b2974d7bac68a19a6155a5f0', 'request_id': 'req-09c5d26a-fbbb-409e-808c-d57829f3e481'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:27.604507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db251662b2974d7bac68a19a6155a5f0', 'request_id': 'req-09c5d26a-fbbb-409e-808c-d57829f3e481'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:27.611155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db251662b2974d7bac68a19a6155a5f0', 'request_id': 'req-09c5d26a-fbbb-409e-808c-d57829f3e481'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:27.627162 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] The token cache is being invalidate because project db251662b2974d7bac68a19a6155a5f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:27.627698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:27.628372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09c5d26a-fbbb-409e-808c-d57829f3e481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:27.630072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 941/1908] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:27 2022] DELETE /identity/v3/projects/db251662b2974d7bac68a19a6155a5f0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:27.637704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935872f-5078-4280-8081-2e04b1b4fc12 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:27.638461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935872f-5078-4280-8081-2e04b1b4fc12 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:27.638461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935872f-5078-4280-8081-2e04b1b4fc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:27.676564 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7935872f-5078-4280-8081-2e04b1b4fc12 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:27.679263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7935872f-5078-4280-8081-2e04b1b4fc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:27.752244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 968/1909] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:27.763558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654b20-0c07-4333-b0bf-0bde796eccd0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:27.764296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654b20-0c07-4333-b0bf-0bde796eccd0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:27.764646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654b20-0c07-4333-b0bf-0bde796eccd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:27.789379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-54654b20-0c07-4333-b0bf-0bde796eccd0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:27.790655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-54654b20-0c07-4333-b0bf-0bde796eccd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:27.798779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 942/1910] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:27.804033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935872f-5078-4280-8081-2e04b1b4fc12 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:27.814561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935872f-5078-4280-8081-2e04b1b4fc12 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:27.819920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8b5ee-434e-47de-a867-0214ba254b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:27.820532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8b5ee-434e-47de-a867-0214ba254b2d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:27.820834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8b5ee-434e-47de-a867-0214ba254b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:27.821106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8b5ee-434e-47de-a867-0214ba254b2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:27.821789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8b5ee-434e-47de-a867-0214ba254b2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:09:27.822224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8b5ee-434e-47de-a867-0214ba254b2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:27.823700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8b5ee-434e-47de-a867-0214ba254b2d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:27.837545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 969/1911] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:09:27.842052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54654b20-0c07-4333-b0bf-0bde796eccd0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:27.858677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54654b20-0c07-4333-b0bf-0bde796eccd0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:27.863784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c7ab1-9d0e-43aa-9c44-7242d362047b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:27.864403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c7ab1-9d0e-43aa-9c44-7242d362047b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:27.864759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c7ab1-9d0e-43aa-9c44-7242d362047b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:27.865031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c7ab1-9d0e-43aa-9c44-7242d362047b admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:27.865687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c7ab1-9d0e-43aa-9c44-7242d362047b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:27.867615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c7ab1-9d0e-43aa-9c44-7242d362047b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:27.899585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 943/1912] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:27 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:27.904018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8b5ee-434e-47de-a867-0214ba254b2d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:27.916735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8b5ee-434e-47de-a867-0214ba254b2d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:27.985311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614edce9-fd89-4a29-bc79-ec1b3b703e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:27.985311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614edce9-fd89-4a29-bc79-ec1b3b703e5c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:27.985311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614edce9-fd89-4a29-bc79-ec1b3b703e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:27.985311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614edce9-fd89-4a29-bc79-ec1b3b703e5c admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:27.985846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614edce9-fd89-4a29-bc79-ec1b3b703e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:27.988097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614edce9-fd89-4a29-bc79-ec1b3b703e5c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:28.016681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 970/1913] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:27 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:28.020588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c7ab1-9d0e-43aa-9c44-7242d362047b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:28.035066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c7ab1-9d0e-43aa-9c44-7242d362047b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:28.053361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed77e4-c7f3-4e89-a950-aca0683a38fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:28.054420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed77e4-c7f3-4e89-a950-aca0683a38fb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:28.054826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed77e4-c7f3-4e89-a950-aca0683a38fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:28.055234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed77e4-c7f3-4e89-a950-aca0683a38fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:28.057794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed77e4-c7f3-4e89-a950-aca0683a38fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:28.059023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed77e4-c7f3-4e89-a950-aca0683a38fb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:28.074660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 944/1914] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:09:28.076680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614edce9-fd89-4a29-bc79-ec1b3b703e5c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:28.089613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614edce9-fd89-4a29-bc79-ec1b3b703e5c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:28.095119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ccd02-2fec-4845-89ff-b6d1e61b80f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:28.095930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ccd02-2fec-4845-89ff-b6d1e61b80f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:28.096213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ccd02-2fec-4845-89ff-b6d1e61b80f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:28.096486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ccd02-2fec-4845-89ff-b6d1e61b80f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:28.097187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ccd02-2fec-4845-89ff-b6d1e61b80f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:28.098929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ccd02-2fec-4845-89ff-b6d1e61b80f6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:28.116359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 971/1915] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:28.120811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed77e4-c7f3-4e89-a950-aca0683a38fb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:28.132549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed77e4-c7f3-4e89-a950-aca0683a38fb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:28.138393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfdb7d-285b-4099-8d70-0c9790e4786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:28.138393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfdb7d-285b-4099-8d70-0c9790e4786e admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:28.138676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfdb7d-285b-4099-8d70-0c9790e4786e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:28.139063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfdb7d-285b-4099-8d70-0c9790e4786e admin admin] PATH_INFO: `/v3/projects/d95cd257ccb745b48422cd4ce8e527c5/users/6ae03636dbf946ab8259f5d9e2848a29/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:28.157307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfdb7d-285b-4099-8d70-0c9790e4786e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d95cd257ccb745b48422cd4ce8e527c5, user_id=6ae03636dbf946ab8259f5d9e2848a29, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:28.160056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfdb7d-285b-4099-8d70-0c9790e4786e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:28.188161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 945/1916] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:28 2022] PUT /identity/v3/projects/d95cd257ccb745b48422cd4ce8e527c5/users/6ae03636dbf946ab8259f5d9e2848a29/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:28.199347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79497ae-34c4-4acc-ad2e-6f26b6be9dd2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:28.200490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79497ae-34c4-4acc-ad2e-6f26b6be9dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:28.200490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79497ae-34c4-4acc-ad2e-6f26b6be9dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:28.252045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d79497ae-34c4-4acc-ad2e-6f26b6be9dd2 None None] MFA Rules not processed for user `6ae03636dbf946ab8259f5d9e2848a29`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:28.254724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d79497ae-34c4-4acc-ad2e-6f26b6be9dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:28.279216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:28.391552 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 972/1917] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:28.406602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfdb7d-285b-4099-8d70-0c9790e4786e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:28.408464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d0cfdb7d-285b-4099-8d70-0c9790e4786e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:28.463482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:28.513550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfdb7d-285b-4099-8d70-0c9790e4786e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:28.517589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6e24f-a029-4b48-8610-0c8ba310a528 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b957b367ddf2483f84056ccec57fc515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:28.518255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6e24f-a029-4b48-8610-0c8ba310a528 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:28.518537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6e24f-a029-4b48-8610-0c8ba310a528 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:28.518815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6e24f-a029-4b48-8610-0c8ba310a528 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:28.529360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6e24f-a029-4b48-8610-0c8ba310a528 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:28.530875 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6e24f-a029-4b48-8610-0c8ba310a528 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:28.544258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 946/1918] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:09:33.440129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79497ae-34c4-4acc-ad2e-6f26b6be9dd2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:33.441842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d79497ae-34c4-4acc-ad2e-6f26b6be9dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:33.516885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79497ae-34c4-4acc-ad2e-6f26b6be9dd2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:33.522159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:33.523221 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:33.523714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:33.524004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] PATH_INFO: `/v3/users/a50e2738b9d541e0bb94948d6fbe0895` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:33.525505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a50e2738b9d541e0bb94948d6fbe0895)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:33.528906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:33.589462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a50e2738b9d541e0bb94948d6fbe0895'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:33.590417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a50e2738b9d541e0bb94948d6fbe0895'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:33.591126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a50e2738b9d541e0bb94948d6fbe0895'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:33.603326 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a50e2738b9d541e0bb94948d6fbe0895'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:33.629600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a50e2738b9d541e0bb94948d6fbe0895'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:33.643639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 973/1919] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:33 2022] DELETE /identity/v3/users/a50e2738b9d541e0bb94948d6fbe0895 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:33.649595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400079e3-b81a-4d12-a0a5-d839d53c5348 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:33.650353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400079e3-b81a-4d12-a0a5-d839d53c5348 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:33.650353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400079e3-b81a-4d12-a0a5-d839d53c5348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:33.670583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-400079e3-b81a-4d12-a0a5-d839d53c5348 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:33.671823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-400079e3-b81a-4d12-a0a5-d839d53c5348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:33.743234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 947/1920] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:33.753352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:33.754807 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:33.796507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:33.826601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435cf558-9116-4995-a6ba-60c018b2f582 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:33.830820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c884477-bed8-4455-b209-6a356c5c0a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:33.831457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c884477-bed8-4455-b209-6a356c5c0a69 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:33.831736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c884477-bed8-4455-b209-6a356c5c0a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:33.832003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c884477-bed8-4455-b209-6a356c5c0a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:33.848950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c884477-bed8-4455-b209-6a356c5c0a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:33.850879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c884477-bed8-4455-b209-6a356c5c0a69 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:33.866729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 974/1921] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:34.158397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400079e3-b81a-4d12-a0a5-d839d53c5348 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:34.160133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-400079e3-b81a-4d12-a0a5-d839d53c5348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:34.175503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400079e3-b81a-4d12-a0a5-d839d53c5348 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:34.183692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:34.184509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:34.184837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:34.185227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] PATH_INFO: `/v3/users/371e98756d4e4623a6c846279a0db0a2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:34.186825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=371e98756d4e4623a6c846279a0db0a2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:34.189231 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:34.278904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '371e98756d4e4623a6c846279a0db0a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.290309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '371e98756d4e4623a6c846279a0db0a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.291628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '371e98756d4e4623a6c846279a0db0a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.293123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '371e98756d4e4623a6c846279a0db0a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.308490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-21448797-5a0d-4453-8f1b-c742e468d092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '371e98756d4e4623a6c846279a0db0a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.318041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c884477-bed8-4455-b209-6a356c5c0a69 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:34.320342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0c884477-bed8-4455-b209-6a356c5c0a69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:34.349697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c884477-bed8-4455-b209-6a356c5c0a69 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:34.379361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:34.386947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:34.388192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:34.388728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] PATH_INFO: `/v3/projects/70f9c2b1ba304396b0908bf122467da3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:34.390503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70f9c2b1ba304396b0908bf122467da3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:34.394037 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:34.400697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 948/1922] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:34 2022] DELETE /identity/v3/users/371e98756d4e4623a6c846279a0db0a2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:34.405178 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f874d81-73ff-4310-9da7-0b463ef487aa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:34.405519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f874d81-73ff-4310-9da7-0b463ef487aa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:34.407798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f874d81-73ff-4310-9da7-0b463ef487aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:34.519447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70f9c2b1ba304396b0908bf122467da3', 'request_id': 'req-376de1b0-f259-472b-aa05-65c22f0ad021'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.540636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9f874d81-73ff-4310-9da7-0b463ef487aa None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:34.549113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9f874d81-73ff-4310-9da7-0b463ef487aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:34.549831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70f9c2b1ba304396b0908bf122467da3', 'request_id': 'req-376de1b0-f259-472b-aa05-65c22f0ad021'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.581598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70f9c2b1ba304396b0908bf122467da3', 'request_id': 'req-376de1b0-f259-472b-aa05-65c22f0ad021'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.640871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] The token cache is being invalidate because project 70f9c2b1ba304396b0908bf122467da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:34.648858 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.650063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.653374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 975/1923] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:34 2022] DELETE /identity/v3/projects/70f9c2b1ba304396b0908bf122467da3 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:34.654533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:34.664199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:34.665898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:34.725445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 949/1924] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:34.835030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376de1b0-f259-472b-aa05-65c22f0ad021 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:34.851492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:34.852492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:34.852879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:34.853261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] PATH_INFO: `/v3/users/351d9c59a837484ab70dbe0e54a25e5e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:34.855308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f874d81-73ff-4310-9da7-0b463ef487aa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:34.855789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=351d9c59a837484ab70dbe0e54a25e5e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:34.857923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9f874d81-73ff-4310-9da7-0b463ef487aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:34.858312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:34.950530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '351d9c59a837484ab70dbe0e54a25e5e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.951691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '351d9c59a837484ab70dbe0e54a25e5e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.952420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '351d9c59a837484ab70dbe0e54a25e5e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.985154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '351d9c59a837484ab70dbe0e54a25e5e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:34.995761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:35.005443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '351d9c59a837484ab70dbe0e54a25e5e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:35.045504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 976/1925] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:34 2022] DELETE /identity/v3/users/351d9c59a837484ab70dbe0e54a25e5e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:35.100749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f874d81-73ff-4310-9da7-0b463ef487aa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:35.201914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad81e6-22aa-4530-81e7-12be8a2da27f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:35.202570 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad81e6-22aa-4530-81e7-12be8a2da27f service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:35.202856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad81e6-22aa-4530-81e7-12be8a2da27f service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:35.203245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad81e6-22aa-4530-81e7-12be8a2da27f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:35.205154 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7bad81e6-22aa-4530-81e7-12be8a2da27f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:35.299855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad81e6-22aa-4530-81e7-12be8a2da27f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:35.301543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad81e6-22aa-4530-81e7-12be8a2da27f service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:35.313922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 950/1926] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:35.527946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:35.530982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:35.553746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b08fc8-d5cb-480e-b23f-4fa2368a8c8d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:35.568368 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:35.569146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:35.569799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:35.569930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] PATH_INFO: `/v3/projects/de1fea58ee9c42e4bdaefd2c6e6984ea` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:35.571275 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1fea58ee9c42e4bdaefd2c6e6984ea)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:35.573507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:35.646977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1fea58ee9c42e4bdaefd2c6e6984ea', 'request_id': 'req-7ba60b06-5f78-4c41-8d09-3d27a4130250'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:35.667605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1fea58ee9c42e4bdaefd2c6e6984ea', 'request_id': 'req-7ba60b06-5f78-4c41-8d09-3d27a4130250'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:35.673399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1fea58ee9c42e4bdaefd2c6e6984ea', 'request_id': 'req-7ba60b06-5f78-4c41-8d09-3d27a4130250'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:35.690586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] The token cache is being invalidate because project de1fea58ee9c42e4bdaefd2c6e6984ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:35.690920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:35.691563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:35.699334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8314108-6965-4ed6-8f71-0a46531912b8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:35.699892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8314108-6965-4ed6-8f71-0a46531912b8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:35.700718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8314108-6965-4ed6-8f71-0a46531912b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:35.704823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 977/1927] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:35 2022] DELETE /identity/v3/projects/de1fea58ee9c42e4bdaefd2c6e6984ea => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:35.723383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e8314108-6965-4ed6-8f71-0a46531912b8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:35.725782 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e8314108-6965-4ed6-8f71-0a46531912b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:35.776289 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:35.795455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:35.797072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:35.813956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 951/1928] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:35.824329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e2385-14f6-48da-b572-73fcad688b99 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:35.824646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e2385-14f6-48da-b572-73fcad688b99 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:35.824959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e2385-14f6-48da-b572-73fcad688b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:35.868392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3e9e2385-14f6-48da-b572-73fcad688b99 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:35.869693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3e9e2385-14f6-48da-b572-73fcad688b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:35.884645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 952/1929] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:35.888916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e2385-14f6-48da-b572-73fcad688b99 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:35.905323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e2385-14f6-48da-b572-73fcad688b99 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:35.910308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d64d4-8b1f-4ecf-a660-39762174d698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:35.910942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d64d4-8b1f-4ecf-a660-39762174d698 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:35.911224 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d64d4-8b1f-4ecf-a660-39762174d698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:35.912073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d64d4-8b1f-4ecf-a660-39762174d698 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:35.912755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d64d4-8b1f-4ecf-a660-39762174d698 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:09:35.913064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d64d4-8b1f-4ecf-a660-39762174d698 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:35.914781 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d64d4-8b1f-4ecf-a660-39762174d698 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:35.921102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba60b06-5f78-4c41-8d09-3d27a4130250 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:35.930346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 953/1930] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:09:35.934271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:35.935115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:35.935439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:35.937804 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] PATH_INFO: `/v3/projects/371ee764c7a84ac09c4217a3eeffe2a8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:35.938032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d64d4-8b1f-4ecf-a660-39762174d698 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:35.938966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=371ee764c7a84ac09c4217a3eeffe2a8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:35.940951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:35.953198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d64d4-8b1f-4ecf-a660-39762174d698 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:35.958951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f3c35-9068-4b7c-b06a-ef84d47d364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:35.960215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f3c35-9068-4b7c-b06a-ef84d47d364d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:35.960666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f3c35-9068-4b7c-b06a-ef84d47d364d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:35.960956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8f3c35-9068-4b7c-b06a-ef84d47d364d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:35.961604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f3c35-9068-4b7c-b06a-ef84d47d364d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:35.964848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8f3c35-9068-4b7c-b06a-ef84d47d364d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:36.021906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 954/1931] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:35 2022] POST /identity/v3/projects => generated 369 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:36.025028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f3c35-9068-4b7c-b06a-ef84d47d364d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:36.046086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '371ee764c7a84ac09c4217a3eeffe2a8', 'request_id': 'req-d3b4a95f-711e-4a60-89cb-622426e0f820'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:36.068216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '371ee764c7a84ac09c4217a3eeffe2a8', 'request_id': 'req-d3b4a95f-711e-4a60-89cb-622426e0f820'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:36.091675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:36.093136 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '371ee764c7a84ac09c4217a3eeffe2a8', 'request_id': 'req-d3b4a95f-711e-4a60-89cb-622426e0f820'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:36.122018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] The token cache is being invalidate because project 371ee764c7a84ac09c4217a3eeffe2a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:36.122317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:36.123023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:36.124786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 978/1932] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:35 2022] DELETE /identity/v3/projects/371ee764c7a84ac09c4217a3eeffe2a8 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:36.128710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:36.131002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:36.149013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8f3c35-9068-4b7c-b06a-ef84d47d364d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:36.154189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4305a28-6859-4a83-9151-fc53f82f498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:36.155142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4305a28-6859-4a83-9151-fc53f82f498c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:36.155909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4305a28-6859-4a83-9151-fc53f82f498c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:36.155909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4305a28-6859-4a83-9151-fc53f82f498c admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:36.156344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4305a28-6859-4a83-9151-fc53f82f498c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:36.158409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4305a28-6859-4a83-9151-fc53f82f498c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:36.187473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 955/1933] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:36 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:36.187955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4305a28-6859-4a83-9151-fc53f82f498c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:36.189712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d4305a28-6859-4a83-9151-fc53f82f498c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:36.216128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4a95f-711e-4a60-89cb-622426e0f820 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:36.219195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4305a28-6859-4a83-9151-fc53f82f498c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:36.225021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:36.225851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:36.226110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1bf6c-147e-4b18-bc01-e943cb9d74b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:36.226251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:36.226408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] PATH_INFO: `/v3/users/a5a853d07b014523b988719632110b23` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:36.227019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1bf6c-147e-4b18-bc01-e943cb9d74b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:36.227399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1bf6c-147e-4b18-bc01-e943cb9d74b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:36.227591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5a853d07b014523b988719632110b23)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:36.227865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1bf6c-147e-4b18-bc01-e943cb9d74b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:36.228625 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1bf6c-147e-4b18-bc01-e943cb9d74b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:36.229624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:36.230752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1bf6c-147e-4b18-bc01-e943cb9d74b5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:36.245848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 956/1934] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:36.249744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1bf6c-147e-4b18-bc01-e943cb9d74b5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:36.264035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1bf6c-147e-4b18-bc01-e943cb9d74b5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:36.271455 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a54c7-77f4-42dc-93a4-ebef0ca4ca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:36.272459 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a54c7-77f4-42dc-93a4-ebef0ca4ca57 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:36.272880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a54c7-77f4-42dc-93a4-ebef0ca4ca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:36.273169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a54c7-77f4-42dc-93a4-ebef0ca4ca57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:36.274029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a54c7-77f4-42dc-93a4-ebef0ca4ca57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:36.276582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a54c7-77f4-42dc-93a4-ebef0ca4ca57 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:36.290000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 957/1935] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:09:36.293481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5a853d07b014523b988719632110b23'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:36.294225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5a853d07b014523b988719632110b23'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:36.294577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a54c7-77f4-42dc-93a4-ebef0ca4ca57 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:36.294991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5a853d07b014523b988719632110b23'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:36.296476 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ea1a54c7-77f4-42dc-93a4-ebef0ca4ca57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:36.305194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5a853d07b014523b988719632110b23'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:36.312319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a54c7-77f4-42dc-93a4-ebef0ca4ca57 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:36.319287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d88fe-b3d8-4a8b-a5d8-6ec99539d891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:36.319966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d88fe-b3d8-4a8b-a5d8-6ec99539d891 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:36.320243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d88fe-b3d8-4a8b-a5d8-6ec99539d891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:36.320519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d88fe-b3d8-4a8b-a5d8-6ec99539d891 admin admin] PATH_INFO: `/v3/projects/1dbada0e205241fa8aafd0b3bfcac050/users/c471d12abce24f2690079f6bc0664a0d/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:36.322536 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9ca4a8d1-d517-4475-8d98-0976707014e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5a853d07b014523b988719632110b23'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:36.339108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 979/1936] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:36 2022] DELETE /identity/v3/users/a5a853d07b014523b988719632110b23 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:36.339935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d88fe-b3d8-4a8b-a5d8-6ec99539d891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dbada0e205241fa8aafd0b3bfcac050, user_id=c471d12abce24f2690079f6bc0664a0d, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:36.343883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d88fe-b3d8-4a8b-a5d8-6ec99539d891 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:36.361462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 958/1937] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:36 2022] PUT /identity/v3/projects/1dbada0e205241fa8aafd0b3bfcac050/users/c471d12abce24f2690079f6bc0664a0d/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:36.373787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c232a8-ff0d-44b0-a7e5-483c71814db6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:36.379219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c232a8-ff0d-44b0-a7e5-483c71814db6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:36.379798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c232a8-ff0d-44b0-a7e5-483c71814db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:36.427954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-23c232a8-ff0d-44b0-a7e5-483c71814db6 None None] MFA Rules not processed for user `c471d12abce24f2690079f6bc0664a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:36.429823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-23c232a8-ff0d-44b0-a7e5-483c71814db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:36.549523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:36.561965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d88fe-b3d8-4a8b-a5d8-6ec99539d891 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:36.562562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 980/1938] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:36.654359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:36.686215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d88fe-b3d8-4a8b-a5d8-6ec99539d891 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:36.712223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a15ee-c718-487f-b59f-35d0cd9daaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:36.712904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a15ee-c718-487f-b59f-35d0cd9daaf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:36.713286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a15ee-c718-487f-b59f-35d0cd9daaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:36.713685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178a15ee-c718-487f-b59f-35d0cd9daaf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:36.714461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a15ee-c718-487f-b59f-35d0cd9daaf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:36.717152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178a15ee-c718-487f-b59f-35d0cd9daaf9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:36.765001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 959/1939] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:36 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:36.766287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c232a8-ff0d-44b0-a7e5-483c71814db6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:36.790888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c232a8-ff0d-44b0-a7e5-483c71814db6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:36.841191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a15ee-c718-487f-b59f-35d0cd9daaf9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:36.844902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-178a15ee-c718-487f-b59f-35d0cd9daaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:36.904680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75fc0c-a343-40ce-b74c-9f7801ee649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:36.905493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75fc0c-a343-40ce-b74c-9f7801ee649a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:36.906971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75fc0c-a343-40ce-b74c-9f7801ee649a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:36.907717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75fc0c-a343-40ce-b74c-9f7801ee649a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:36.909970 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75fc0c-a343-40ce-b74c-9f7801ee649a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:36.913844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75fc0c-a343-40ce-b74c-9f7801ee649a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:36.961673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 981/1940] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:36 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:36.965917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75fc0c-a343-40ce-b74c-9f7801ee649a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:36.979870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75fc0c-a343-40ce-b74c-9f7801ee649a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:36.984660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178a15ee-c718-487f-b59f-35d0cd9daaf9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:36.986278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35922f-7c40-4b86-9091-4c7e114df952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:36.990936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:36.991740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:36.991978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35922f-7c40-4b86-9091-4c7e114df952 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:36.992146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:36.992320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35922f-7c40-4b86-9091-4c7e114df952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:36.992470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] PATH_INFO: `/v3/projects/9ebf6097b7f741b5830c749db9bd40a4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:36.992623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35922f-7c40-4b86-9091-4c7e114df952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:36.993203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35922f-7c40-4b86-9091-4c7e114df952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:36.993597 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ebf6097b7f741b5830c749db9bd40a4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:36.995157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35922f-7c40-4b86-9091-4c7e114df952 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:36.997663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:37.013219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 982/1941] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:37.014462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35922f-7c40-4b86-9091-4c7e114df952 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:37.026049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35922f-7c40-4b86-9091-4c7e114df952 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:37.038166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938f569-3214-4e5b-957a-d4916a7bbda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:37.038875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938f569-3214-4e5b-957a-d4916a7bbda4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.039159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938f569-3214-4e5b-957a-d4916a7bbda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.040244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938f569-3214-4e5b-957a-d4916a7bbda4 admin admin] PATH_INFO: `/v3/projects/4b0fe5ae59364755a6d95b1309368154/users/cb13f6e8c64943ac86cff66b54781730/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.064129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938f569-3214-4e5b-957a-d4916a7bbda4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0fe5ae59364755a6d95b1309368154, user_id=cb13f6e8c64943ac86cff66b54781730, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:37.068978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b938f569-3214-4e5b-957a-d4916a7bbda4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:37.086982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ebf6097b7f741b5830c749db9bd40a4', 'request_id': 'req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:37.096750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 983/1942] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:37 2022] PUT /identity/v3/projects/4b0fe5ae59364755a6d95b1309368154/users/cb13f6e8c64943ac86cff66b54781730/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:37.099920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938f569-3214-4e5b-957a-d4916a7bbda4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:37.105160 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ebf6097b7f741b5830c749db9bd40a4', 'request_id': 'req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:37.114285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ebf6097b7f741b5830c749db9bd40a4', 'request_id': 'req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:37.142166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] The token cache is being invalidate because project 9ebf6097b7f741b5830c749db9bd40a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:37.142905 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:37.143892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f7e9e3f0-5a7a-4ba9-8d85-b04512e5cea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:37.146267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 960/1943] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:36 2022] DELETE /identity/v3/projects/9ebf6097b7f741b5830c749db9bd40a4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:37.155761 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762f8a6-58be-4f89-8da1-a54694d17cae None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.156257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762f8a6-58be-4f89-8da1-a54694d17cae None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.156895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762f8a6-58be-4f89-8da1-a54694d17cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.187146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5762f8a6-58be-4f89-8da1-a54694d17cae None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:37.188573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5762f8a6-58be-4f89-8da1-a54694d17cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:37.214824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938f569-3214-4e5b-957a-d4916a7bbda4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:37.220801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdc9d6-b2f1-4aed-9bd0-6e9653eddefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:37.221515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdc9d6-b2f1-4aed-9bd0-6e9653eddefd admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.221889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdc9d6-b2f1-4aed-9bd0-6e9653eddefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.222167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdc9d6-b2f1-4aed-9bd0-6e9653eddefd admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.223000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdc9d6-b2f1-4aed-9bd0-6e9653eddefd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:37.225214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdc9d6-b2f1-4aed-9bd0-6e9653eddefd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:37.236067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 984/1944] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:37.241705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdc9d6-b2f1-4aed-9bd0-6e9653eddefd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:37.243953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cebdc9d6-b2f1-4aed-9bd0-6e9653eddefd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:37.249005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:37.259338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdc9d6-b2f1-4aed-9bd0-6e9653eddefd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:37.262425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 961/1945] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:37.268456 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f1311-bdf7-4d5b-acfc-7f2b7804eeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:37.268925 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f1311-bdf7-4d5b-acfc-7f2b7804eeaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.269214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f1311-bdf7-4d5b-acfc-7f2b7804eeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.269491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f1311-bdf7-4d5b-acfc-7f2b7804eeaa admin admin] PATH_INFO: `/v3/projects/4b0fe5ae59364755a6d95b1309368154/users/cb13f6e8c64943ac86cff66b54781730/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.270265 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3b6d6-a389-49da-acec-1c7f3795fbb1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.270669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3b6d6-a389-49da-acec-1c7f3795fbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.271709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3b6d6-a389-49da-acec-1c7f3795fbb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.271811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f1311-bdf7-4d5b-acfc-7f2b7804eeaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0fe5ae59364755a6d95b1309368154, user_id=cb13f6e8c64943ac86cff66b54781730, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:37.274727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f1311-bdf7-4d5b-acfc-7f2b7804eeaa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:37.287078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 985/1946] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:37 2022] PUT /identity/v3/projects/4b0fe5ae59364755a6d95b1309368154/users/cb13f6e8c64943ac86cff66b54781730/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:37.298365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c92f1-7de0-49cd-9a1c-5848389b6cc9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.298653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c92f1-7de0-49cd-9a1c-5848389b6cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.298956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c92f1-7de0-49cd-9a1c-5848389b6cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.299721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-83a3b6d6-a389-49da-acec-1c7f3795fbb1 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:37.301045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83a3b6d6-a389-49da-acec-1c7f3795fbb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:37.331451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-824c92f1-7de0-49cd-9a1c-5848389b6cc9 None None] MFA Rules not processed for user `cb13f6e8c64943ac86cff66b54781730`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:37.332738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-824c92f1-7de0-49cd-9a1c-5848389b6cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:37.362243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:37.378406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 962/1947] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:37.379494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3b6d6-a389-49da-acec-1c7f3795fbb1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:37.392749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3b6d6-a389-49da-acec-1c7f3795fbb1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:37.401680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94def9-5f49-4ffd-b577-008645c6cdd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:37.402421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94def9-5f49-4ffd-b577-008645c6cdd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.402727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94def9-5f49-4ffd-b577-008645c6cdd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.403006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94def9-5f49-4ffd-b577-008645c6cdd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.403857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94def9-5f49-4ffd-b577-008645c6cdd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:09:37.404173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94def9-5f49-4ffd-b577-008645c6cdd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:37.405873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94def9-5f49-4ffd-b577-008645c6cdd9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:37.414581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 986/1948] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:37.419982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 963/1949] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:09:37.426641 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c92f1-7de0-49cd-9a1c-5848389b6cc9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:37.431517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94def9-5f49-4ffd-b577-008645c6cdd9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:37.434817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cf94def9-5f49-4ffd-b577-008645c6cdd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:37.441865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c92f1-7de0-49cd-9a1c-5848389b6cc9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:37.449478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9e786-3b9d-4728-b9c5-4f71082911af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:37.450763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9e786-3b9d-4728-b9c5-4f71082911af admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.451454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9e786-3b9d-4728-b9c5-4f71082911af admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.451957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f9e786-3b9d-4728-b9c5-4f71082911af admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.452967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9e786-3b9d-4728-b9c5-4f71082911af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:37.455175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f9e786-3b9d-4728-b9c5-4f71082911af admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:37.499196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 987/1950] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:37 2022] POST /identity/v3/projects => generated 367 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:37.504298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9e786-3b9d-4728-b9c5-4f71082911af admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:37.531447 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94def9-5f49-4ffd-b577-008645c6cdd9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:37.604490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f9e786-3b9d-4728-b9c5-4f71082911af admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:37.617049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993ef27-0775-4e95-a8f0-1e720ad6c326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:37.618145 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993ef27-0775-4e95-a8f0-1e720ad6c326 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.618820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993ef27-0775-4e95-a8f0-1e720ad6c326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.619327 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993ef27-0775-4e95-a8f0-1e720ad6c326 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.620904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993ef27-0775-4e95-a8f0-1e720ad6c326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:37.626927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993ef27-0775-4e95-a8f0-1e720ad6c326 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:37.649085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2b9b5-b30c-4d5e-94dc-32da6df27601 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:37.650960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2b9b5-b30c-4d5e-94dc-32da6df27601 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.651631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2b9b5-b30c-4d5e-94dc-32da6df27601 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.652251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2b9b5-b30c-4d5e-94dc-32da6df27601 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.674109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 988/1951] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:37 2022] POST /identity/v3/users => generated 284 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:37.679862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993ef27-0775-4e95-a8f0-1e720ad6c326 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:37.694564 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993ef27-0775-4e95-a8f0-1e720ad6c326 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:37.730611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b858f-ab8d-42f3-b816-be6cb32da326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:37.731083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:37.733298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b858f-ab8d-42f3-b816-be6cb32da326 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.734119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b858f-ab8d-42f3-b816-be6cb32da326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.736413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b858f-ab8d-42f3-b816-be6cb32da326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.736977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b858f-ab8d-42f3-b816-be6cb32da326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:37.742751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b858f-ab8d-42f3-b816-be6cb32da326 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:37.747292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2b9b5-b30c-4d5e-94dc-32da6df27601 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:37.748912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2b9b5-b30c-4d5e-94dc-32da6df27601 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:37.768977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 964/1952] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:09:37.808024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 989/1953] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:09:37.812276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2b9b5-b30c-4d5e-94dc-32da6df27601 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:37.859968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2b9b5-b30c-4d5e-94dc-32da6df27601 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:37.875705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe20a2b-c17b-4632-851c-bc28ee94a193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:37.877067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe20a2b-c17b-4632-851c-bc28ee94a193 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.877801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe20a2b-c17b-4632-851c-bc28ee94a193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.878129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe20a2b-c17b-4632-851c-bc28ee94a193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.878900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe20a2b-c17b-4632-851c-bc28ee94a193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:37.881387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe20a2b-c17b-4632-851c-bc28ee94a193 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:37.901807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 965/1954] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:37.906370 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b858f-ab8d-42f3-b816-be6cb32da326 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:37.920118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b858f-ab8d-42f3-b816-be6cb32da326 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:37.941950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d611f3-c088-4062-943e-9ceea37f3e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:37.943038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d611f3-c088-4062-943e-9ceea37f3e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:37.943390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d611f3-c088-4062-943e-9ceea37f3e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:37.943694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d611f3-c088-4062-943e-9ceea37f3e1a admin admin] PATH_INFO: `/v3/projects/242ec8a2f75a4b0c96829dc4a1f7cb53/users/f99a876e8f0949f69593767d15be59ec/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:37.984380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d611f3-c088-4062-943e-9ceea37f3e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=242ec8a2f75a4b0c96829dc4a1f7cb53, user_id=f99a876e8f0949f69593767d15be59ec, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:37.991379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d611f3-c088-4062-943e-9ceea37f3e1a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:38.013922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 990/1955] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:37 2022] PUT /identity/v3/projects/242ec8a2f75a4b0c96829dc4a1f7cb53/users/f99a876e8f0949f69593767d15be59ec/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:38.035711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcc213-2a74-4302-89c9-b5b6e4034d80 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:38.035711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcc213-2a74-4302-89c9-b5b6e4034d80 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:38.035711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbcc213-2a74-4302-89c9-b5b6e4034d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:38.076307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3bbcc213-2a74-4302-89c9-b5b6e4034d80 None None] MFA Rules not processed for user `f99a876e8f0949f69593767d15be59ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:38.079141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3bbcc213-2a74-4302-89c9-b5b6e4034d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:38.276181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 966/1956] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:38.293324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d611f3-c088-4062-943e-9ceea37f3e1a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:38.296470 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c5d611f3-c088-4062-943e-9ceea37f3e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:38.443245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d611f3-c088-4062-943e-9ceea37f3e1a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:38.451012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf472d-e666-4996-8b4c-a88083399166 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b957b367ddf2483f84056ccec57fc515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:38.452081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf472d-e666-4996-8b4c-a88083399166 service glance] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:38.452381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf472d-e666-4996-8b4c-a88083399166 service glance] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:38.452930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf472d-e666-4996-8b4c-a88083399166 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:38.497833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf472d-e666-4996-8b4c-a88083399166 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:38.497833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaf472d-e666-4996-8b4c-a88083399166 service glance] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:38.503889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:38.527548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 991/1957] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:40.819457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcc213-2a74-4302-89c9-b5b6e4034d80 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:40.937926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbcc213-2a74-4302-89c9-b5b6e4034d80 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:40.948430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:40.949493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:40.950217 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:40.950696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] PATH_INFO: `/v3/users/c471d12abce24f2690079f6bc0664a0d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:40.953161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c471d12abce24f2690079f6bc0664a0d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:40.955377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:41.051144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c471d12abce24f2690079f6bc0664a0d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:41.070116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c471d12abce24f2690079f6bc0664a0d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:41.071062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c471d12abce24f2690079f6bc0664a0d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:41.074252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c471d12abce24f2690079f6bc0664a0d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:41.091004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c471d12abce24f2690079f6bc0664a0d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:41.104497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:41.129882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 967/1958] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:40 2022] DELETE /identity/v3/users/c471d12abce24f2690079f6bc0664a0d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:41.134889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4878f5-a7bb-4b40-8675-ca3205b6e5d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:41.135451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4878f5-a7bb-4b40-8675-ca3205b6e5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:41.135451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4878f5-a7bb-4b40-8675-ca3205b6e5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:41.165661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ca4878f5-a7bb-4b40-8675-ca3205b6e5d5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:41.168371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ca4878f5-a7bb-4b40-8675-ca3205b6e5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:41.356751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 992/1959] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:41.381438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:41.383211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:41.513061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659832bc-a8d7-43b1-8481-bcda3cca1443 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:41.522092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a3c01-f345-464d-be43-3d95ef22fd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:41.529542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a3c01-f345-464d-be43-3d95ef22fd2b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:41.529975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a3c01-f345-464d-be43-3d95ef22fd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:41.530262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a3c01-f345-464d-be43-3d95ef22fd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:41.544383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a3c01-f345-464d-be43-3d95ef22fd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:41.554338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a3c01-f345-464d-be43-3d95ef22fd2b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:41.569485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 968/1960] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:42.485566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4878f5-a7bb-4b40-8675-ca3205b6e5d5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:42.491707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ca4878f5-a7bb-4b40-8675-ca3205b6e5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:42.529293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4878f5-a7bb-4b40-8675-ca3205b6e5d5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:42.558654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:42.559377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:42.559695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:42.559998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] PATH_INFO: `/v3/projects/1dbada0e205241fa8aafd0b3bfcac050` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:42.571047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dbada0e205241fa8aafd0b3bfcac050)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:42.573514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:42.690590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dbada0e205241fa8aafd0b3bfcac050', 'request_id': 'req-09b3ff10-1d4b-447e-ac8d-f273b58a8968'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:42.711185 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dbada0e205241fa8aafd0b3bfcac050', 'request_id': 'req-09b3ff10-1d4b-447e-ac8d-f273b58a8968'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:42.727018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dbada0e205241fa8aafd0b3bfcac050', 'request_id': 'req-09b3ff10-1d4b-447e-ac8d-f273b58a8968'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:42.760189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] The token cache is being invalidate because project 1dbada0e205241fa8aafd0b3bfcac050 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:42.760576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:42.768354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:42.779040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 993/1961] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:42 2022] DELETE /identity/v3/projects/1dbada0e205241fa8aafd0b3bfcac050 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:42.786096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a3c01-f345-464d-be43-3d95ef22fd2b service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:42.798077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a3c01-f345-464d-be43-3d95ef22fd2b service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:42.924710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:42.925342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:42.925627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:42.925929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] PATH_INFO: `/v3/users/cb13f6e8c64943ac86cff66b54781730` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:42.928064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb13f6e8c64943ac86cff66b54781730)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:42.934011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:43.054360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb13f6e8c64943ac86cff66b54781730'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:43.068388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb13f6e8c64943ac86cff66b54781730'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:43.069298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb13f6e8c64943ac86cff66b54781730'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:43.070119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb13f6e8c64943ac86cff66b54781730'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:43.080325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e7377390-dbc2-4e56-9c04-b7e7490652d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb13f6e8c64943ac86cff66b54781730'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:43.109482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 969/1962] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:42 2022] DELETE /identity/v3/users/cb13f6e8c64943ac86cff66b54781730 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:44.394278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:44.396093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:44.419506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3ff10-1d4b-447e-ac8d-f273b58a8968 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:44.519356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:44.520188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:44.520504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:44.521063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] PATH_INFO: `/v3/projects/4b0fe5ae59364755a6d95b1309368154` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:44.522279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b0fe5ae59364755a6d95b1309368154)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:44.524471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:44.561582 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:44.598364 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b0fe5ae59364755a6d95b1309368154', 'request_id': 'req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:44.622075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b0fe5ae59364755a6d95b1309368154', 'request_id': 'req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:44.628852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b0fe5ae59364755a6d95b1309368154', 'request_id': 'req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:44.642048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] The token cache is being invalidate because project 4b0fe5ae59364755a6d95b1309368154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:44.642801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:44.643746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e0772163-8ee9-4877-a27e-3b8a6d82b1d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:44.645844 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 994/1963] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:44 2022] DELETE /identity/v3/projects/4b0fe5ae59364755a6d95b1309368154 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:44.651077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa8da3-1fc1-4237-a075-bd7d08281b4e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:44.652018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa8da3-1fc1-4237-a075-bd7d08281b4e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:44.652492 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa8da3-1fc1-4237-a075-bd7d08281b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:44.679049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bbaa8da3-1fc1-4237-a075-bd7d08281b4e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:44.680547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bbaa8da3-1fc1-4237-a075-bd7d08281b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:44.687288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:44.781862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 970/1964] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:09:44.790806 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49b11a-1bd0-46d1-a3e9-03661e69dd13 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:44.791130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49b11a-1bd0-46d1-a3e9-03661e69dd13 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:44.791420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49b11a-1bd0-46d1-a3e9-03661e69dd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:44.827875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cf49b11a-1bd0-46d1-a3e9-03661e69dd13 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:44.835697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cf49b11a-1bd0-46d1-a3e9-03661e69dd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:44.846236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 995/1965] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:44.850368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa8da3-1fc1-4237-a075-bd7d08281b4e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:44.868699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa8da3-1fc1-4237-a075-bd7d08281b4e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:44.873926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faf172-9b97-4651-81fe-82283102f3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:44.874575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faf172-9b97-4651-81fe-82283102f3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:44.874998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faf172-9b97-4651-81fe-82283102f3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:44.875130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faf172-9b97-4651-81fe-82283102f3f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:44.875812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6faf172-9b97-4651-81fe-82283102f3f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:09:44.876119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6faf172-9b97-4651-81fe-82283102f3f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:44.881730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6faf172-9b97-4651-81fe-82283102f3f9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:44.907607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 971/1966] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:09:44.913712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49b11a-1bd0-46d1-a3e9-03661e69dd13 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:44.934355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49b11a-1bd0-46d1-a3e9-03661e69dd13 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:44.947320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab47c9-99d0-4c89-a364-e695f363aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:44.947973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab47c9-99d0-4c89-a364-e695f363aa6d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:44.948255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab47c9-99d0-4c89-a364-e695f363aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:44.948675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab47c9-99d0-4c89-a364-e695f363aa6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:44.949328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab47c9-99d0-4c89-a364-e695f363aa6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:44.952144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab47c9-99d0-4c89-a364-e695f363aa6d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:45.027869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 996/1967] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:44 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:45.032924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faf172-9b97-4651-81fe-82283102f3f9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:45.045873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faf172-9b97-4651-81fe-82283102f3f9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:45.165518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93669475-230b-4f78-ad38-a63c61b07a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:45.166393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93669475-230b-4f78-ad38-a63c61b07a10 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.166688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93669475-230b-4f78-ad38-a63c61b07a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.167450 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93669475-230b-4f78-ad38-a63c61b07a10 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.170006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93669475-230b-4f78-ad38-a63c61b07a10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:45.172199 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93669475-230b-4f78-ad38-a63c61b07a10 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:45.217798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 972/1968] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:45 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:45.219627 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab47c9-99d0-4c89-a364-e695f363aa6d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:45.234801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab47c9-99d0-4c89-a364-e695f363aa6d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:45.241525 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9cc89-dcfb-4670-9afd-0b98062fe108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:45.242389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9cc89-dcfb-4670-9afd-0b98062fe108 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.242819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9cc89-dcfb-4670-9afd-0b98062fe108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.243377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9cc89-dcfb-4670-9afd-0b98062fe108 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.244141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9cc89-dcfb-4670-9afd-0b98062fe108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:45.246024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9cc89-dcfb-4670-9afd-0b98062fe108 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:45.261731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 997/1969] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:45.263228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93669475-230b-4f78-ad38-a63c61b07a10 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:45.276775 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93669475-230b-4f78-ad38-a63c61b07a10 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:45.283710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87566f82-5288-466b-8728-766d17161645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:45.284836 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87566f82-5288-466b-8728-766d17161645 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.285336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87566f82-5288-466b-8728-766d17161645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.285693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87566f82-5288-466b-8728-766d17161645 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.287302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87566f82-5288-466b-8728-766d17161645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:45.289791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87566f82-5288-466b-8728-766d17161645 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:45.302490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 973/1970] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:45.307429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9cc89-dcfb-4670-9afd-0b98062fe108 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:45.319593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9cc89-dcfb-4670-9afd-0b98062fe108 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:45.324621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28c511-97d0-4fef-b4fc-56cddbc75942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:45.325380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28c511-97d0-4fef-b4fc-56cddbc75942 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.325903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28c511-97d0-4fef-b4fc-56cddbc75942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.326242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d28c511-97d0-4fef-b4fc-56cddbc75942 admin admin] PATH_INFO: `/v3/projects/6c6fbfe7ffa54f9baa3ffe57030afe04/users/98fe7d3b29ea4a329362cfd5cbefa7f9/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.342539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d28c511-97d0-4fef-b4fc-56cddbc75942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c6fbfe7ffa54f9baa3ffe57030afe04, user_id=98fe7d3b29ea4a329362cfd5cbefa7f9, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:45.345384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d28c511-97d0-4fef-b4fc-56cddbc75942 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:45.369237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 998/1971] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:45 2022] PUT /identity/v3/projects/6c6fbfe7ffa54f9baa3ffe57030afe04/users/98fe7d3b29ea4a329362cfd5cbefa7f9/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:45.381881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e009d69-e4b2-480d-83d6-8f74622ba723 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.382289 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e009d69-e4b2-480d-83d6-8f74622ba723 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.383467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e009d69-e4b2-480d-83d6-8f74622ba723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.415640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3e009d69-e4b2-480d-83d6-8f74622ba723 None None] MFA Rules not processed for user `98fe7d3b29ea4a329362cfd5cbefa7f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:45.417232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3e009d69-e4b2-480d-83d6-8f74622ba723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:45.496109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 974/1972] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:45.497273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28c511-97d0-4fef-b4fc-56cddbc75942 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:45.566169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d28c511-97d0-4fef-b4fc-56cddbc75942 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:45.571573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f62b51-54ac-4753-9bb1-f49029e5fd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:45.572335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f62b51-54ac-4753-9bb1-f49029e5fd59 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.572497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f62b51-54ac-4753-9bb1-f49029e5fd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.572773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f62b51-54ac-4753-9bb1-f49029e5fd59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.573454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f62b51-54ac-4753-9bb1-f49029e5fd59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:45.575551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f62b51-54ac-4753-9bb1-f49029e5fd59 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:45.604258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 999/1973] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:45 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:45.608243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e009d69-e4b2-480d-83d6-8f74622ba723 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:45.619481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e009d69-e4b2-480d-83d6-8f74622ba723 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:45.623677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:45.701879 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09384979-319b-462c-ba6c-5726cf9a842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:45.702745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09384979-319b-462c-ba6c-5726cf9a842c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.703116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09384979-319b-462c-ba6c-5726cf9a842c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.703449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09384979-319b-462c-ba6c-5726cf9a842c admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.704318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09384979-319b-462c-ba6c-5726cf9a842c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:45.710914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09384979-319b-462c-ba6c-5726cf9a842c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:45.732155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 975/1974] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:45 2022] POST /identity/v3/users => generated 293 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:45.736629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f62b51-54ac-4753-9bb1-f49029e5fd59 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:45.748401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f62b51-54ac-4753-9bb1-f49029e5fd59 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:45.753667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fb002-5c9f-4ecf-b5ad-44ce315de6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:45.754433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fb002-5c9f-4ecf-b5ad-44ce315de6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.754836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fb002-5c9f-4ecf-b5ad-44ce315de6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.755117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fb002-5c9f-4ecf-b5ad-44ce315de6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.756242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fb002-5c9f-4ecf-b5ad-44ce315de6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:45.758021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203fb002-5c9f-4ecf-b5ad-44ce315de6e6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:45.770041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1000/1975] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:45.775007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09384979-319b-462c-ba6c-5726cf9a842c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:45.785051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09384979-319b-462c-ba6c-5726cf9a842c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:45.791400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be8be1-4ddc-4263-bac7-52d222aa461f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:45.791952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be8be1-4ddc-4263-bac7-52d222aa461f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.792233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be8be1-4ddc-4263-bac7-52d222aa461f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.792507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be8be1-4ddc-4263-bac7-52d222aa461f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.793157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be8be1-4ddc-4263-bac7-52d222aa461f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:45.794804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9be8be1-4ddc-4263-bac7-52d222aa461f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:45.807484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 976/1976] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:45.811595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fb002-5c9f-4ecf-b5ad-44ce315de6e6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:45.822824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fb002-5c9f-4ecf-b5ad-44ce315de6e6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:45.827681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e5e85-89ed-40c5-b4f2-433288a90ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:45.828372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e5e85-89ed-40c5-b4f2-433288a90ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.828653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e5e85-89ed-40c5-b4f2-433288a90ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.828928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e5e85-89ed-40c5-b4f2-433288a90ac8 admin admin] PATH_INFO: `/v3/projects/1cc690dd898d4d13a74473a7dccb2875/users/33049581450e47a9aae57961ed5e670a/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.845727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e5e85-89ed-40c5-b4f2-433288a90ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc690dd898d4d13a74473a7dccb2875, user_id=33049581450e47a9aae57961ed5e670a, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:45.848473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e5e85-89ed-40c5-b4f2-433288a90ac8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:45.858215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:45.864832 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1001/1977] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:45 2022] PUT /identity/v3/projects/1cc690dd898d4d13a74473a7dccb2875/users/33049581450e47a9aae57961ed5e670a/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:45.876431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f1b1-0aa8-462c-9faf-eca2cf60ca89 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:45.876722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f1b1-0aa8-462c-9faf-eca2cf60ca89 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:45.877028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686f1b1-0aa8-462c-9faf-eca2cf60ca89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:45.916350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b686f1b1-0aa8-462c-9faf-eca2cf60ca89 None None] MFA Rules not processed for user `33049581450e47a9aae57961ed5e670a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:45.917608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b686f1b1-0aa8-462c-9faf-eca2cf60ca89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:45.984775 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 977/1978] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:45.994088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e5e85-89ed-40c5-b4f2-433288a90ac8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:46.097348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e5e85-89ed-40c5-b4f2-433288a90ac8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:46.103599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1e594-461a-487b-afa2-5910fa552033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:46.104404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1e594-461a-487b-afa2-5910fa552033 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:46.104780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1e594-461a-487b-afa2-5910fa552033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:46.105205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1e594-461a-487b-afa2-5910fa552033 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:46.106410 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1e594-461a-487b-afa2-5910fa552033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:46.108059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1e594-461a-487b-afa2-5910fa552033 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:46.157833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1002/1979] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:09:45 2022] POST /identity/v3/projects => generated 387 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:09:46.161382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686f1b1-0aa8-462c-9faf-eca2cf60ca89 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:46.173267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686f1b1-0aa8-462c-9faf-eca2cf60ca89 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:46.260306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c25ad-30db-4b35-8187-a56aa23fa1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:46.261102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c25ad-30db-4b35-8187-a56aa23fa1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:46.261475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c25ad-30db-4b35-8187-a56aa23fa1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:46.261902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411c25ad-30db-4b35-8187-a56aa23fa1b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:46.262650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c25ad-30db-4b35-8187-a56aa23fa1b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:46.265572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411c25ad-30db-4b35-8187-a56aa23fa1b8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:46.305111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 978/1980] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:09:46 2022] POST /identity/v3/users => generated 300 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:09:46.310445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1e594-461a-487b-afa2-5910fa552033 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:46.333819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1e594-461a-487b-afa2-5910fa552033 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:46.341642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d515a2-54cc-45f0-82da-1ac0331eee15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:46.342735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d515a2-54cc-45f0-82da-1ac0331eee15 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:46.343226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d515a2-54cc-45f0-82da-1ac0331eee15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:46.343711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d515a2-54cc-45f0-82da-1ac0331eee15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:46.344863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d515a2-54cc-45f0-82da-1ac0331eee15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:46.348065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d515a2-54cc-45f0-82da-1ac0331eee15 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:46.363613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1003/1981] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:09:46.370348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c25ad-30db-4b35-8187-a56aa23fa1b8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:46.382476 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:46.390456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411c25ad-30db-4b35-8187-a56aa23fa1b8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:46.400222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28e98d-48a9-4dc8-a884-10c7ee883cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:46.401949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28e98d-48a9-4dc8-a884-10c7ee883cde admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:46.402811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28e98d-48a9-4dc8-a884-10c7ee883cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:46.403468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28e98d-48a9-4dc8-a884-10c7ee883cde admin admin] PATH_INFO: `/v3/projects/05798f1e62ef4eb3b8f87a5225e80317/users/da700bc1bc8d45848933264e31a95904/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:46.432443 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28e98d-48a9-4dc8-a884-10c7ee883cde admin admin] RBAC: Authorizing `identity:create_grant(project_id=05798f1e62ef4eb3b8f87a5225e80317, user_id=da700bc1bc8d45848933264e31a95904, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:46.435371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28e98d-48a9-4dc8-a884-10c7ee883cde admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:46.456813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 979/1982] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:46 2022] PUT /identity/v3/projects/05798f1e62ef4eb3b8f87a5225e80317/users/da700bc1bc8d45848933264e31a95904/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:46.462785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d515a2-54cc-45f0-82da-1ac0331eee15 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:46.474650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d515a2-54cc-45f0-82da-1ac0331eee15 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:46.566539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989977ca-7933-4910-9a9f-f645daa0c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:46.567248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989977ca-7933-4910-9a9f-f645daa0c5eb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:46.567492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989977ca-7933-4910-9a9f-f645daa0c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:46.567778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989977ca-7933-4910-9a9f-f645daa0c5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:46.568447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989977ca-7933-4910-9a9f-f645daa0c5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:46.570601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989977ca-7933-4910-9a9f-f645daa0c5eb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:46.589969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1004/1983] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:09:46.594548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28e98d-48a9-4dc8-a884-10c7ee883cde admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:46.609156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28e98d-48a9-4dc8-a884-10c7ee883cde admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:46.614011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0e13-5f76-4781-8b13-f287c95d90e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:46.614838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0e13-5f76-4781-8b13-f287c95d90e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:46.615215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0e13-5f76-4781-8b13-f287c95d90e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:46.615512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f0e13-5f76-4781-8b13-f287c95d90e7 admin admin] PATH_INFO: `/v3/projects/05798f1e62ef4eb3b8f87a5225e80317/users/da700bc1bc8d45848933264e31a95904/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:46.617501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f0e13-5f76-4781-8b13-f287c95d90e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05798f1e62ef4eb3b8f87a5225e80317, user_id=da700bc1bc8d45848933264e31a95904, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:46.620207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f0e13-5f76-4781-8b13-f287c95d90e7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:46.631613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 980/1984] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:09:46 2022] PUT /identity/v3/projects/05798f1e62ef4eb3b8f87a5225e80317/users/da700bc1bc8d45848933264e31a95904/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:46.642423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa14e3-a96d-43ff-973c-d0181878b6ba None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:46.642711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa14e3-a96d-43ff-973c-d0181878b6ba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:46.642994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa14e3-a96d-43ff-973c-d0181878b6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:46.675656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7efa14e3-a96d-43ff-973c-d0181878b6ba None None] MFA Rules not processed for user `da700bc1bc8d45848933264e31a95904`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:46.679915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7efa14e3-a96d-43ff-973c-d0181878b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:46.758418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:46.774340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1005/1985] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:46.785206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0e13-5f76-4781-8b13-f287c95d90e7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:46.787569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-105f0e13-5f76-4781-8b13-f287c95d90e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:46.879725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f0e13-5f76-4781-8b13-f287c95d90e7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:46.888488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88c115-f2eb-433a-b6ae-89a51249a5c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:46.890003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88c115-f2eb-433a-b6ae-89a51249a5c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:46.890286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88c115-f2eb-433a-b6ae-89a51249a5c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:46.890602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88c115-f2eb-433a-b6ae-89a51249a5c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:46.913645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88c115-f2eb-433a-b6ae-89a51249a5c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:46.917794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88c115-f2eb-433a-b6ae-89a51249a5c0 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:46.935856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 981/1986] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:47.684819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa14e3-a96d-43ff-973c-d0181878b6ba None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:47.686448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7efa14e3-a96d-43ff-973c-d0181878b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:47.701179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa14e3-a96d-43ff-973c-d0181878b6ba None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:47.705952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2ab3b-065a-4872-b684-b588c43b5232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:47.708439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2ab3b-065a-4872-b684-b588c43b5232 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:47.708439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2ab3b-065a-4872-b684-b588c43b5232 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:47.708439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2ab3b-065a-4872-b684-b588c43b5232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:47.821088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2ab3b-065a-4872-b684-b588c43b5232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:47.822916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2ab3b-065a-4872-b684-b588c43b5232 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:47.845794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1006/1987] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:54.353195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88c115-f2eb-433a-b6ae-89a51249a5c0 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:54.362560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88c115-f2eb-433a-b6ae-89a51249a5c0 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:54.366982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba79301-2dec-49b1-b9ab-bb6b30deab4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:54.367607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba79301-2dec-49b1-b9ab-bb6b30deab4c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:54.367882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba79301-2dec-49b1-b9ab-bb6b30deab4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:54.368155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba79301-2dec-49b1-b9ab-bb6b30deab4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:54.437915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba79301-2dec-49b1-b9ab-bb6b30deab4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:54.439372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba79301-2dec-49b1-b9ab-bb6b30deab4c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:54.450506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 982/1988] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:09:57.093608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2ab3b-065a-4872-b684-b588c43b5232 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:57.188653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2ab3b-065a-4872-b684-b588c43b5232 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:57.197803 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:57.198670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:57.199052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:57.199461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] PATH_INFO: `/v3/users/98fe7d3b29ea4a329362cfd5cbefa7f9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:57.200704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=98fe7d3b29ea4a329362cfd5cbefa7f9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:57.202788 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:57.269361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98fe7d3b29ea4a329362cfd5cbefa7f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:57.270068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98fe7d3b29ea4a329362cfd5cbefa7f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:57.270799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98fe7d3b29ea4a329362cfd5cbefa7f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:57.283312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98fe7d3b29ea4a329362cfd5cbefa7f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:57.299399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98fe7d3b29ea4a329362cfd5cbefa7f9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:57.314980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1007/1989] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:57 2022] DELETE /identity/v3/users/98fe7d3b29ea4a329362cfd5cbefa7f9 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:57.321535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058d2a7-dbb1-41d0-81cf-4864d64f6ec1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:57.322102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058d2a7-dbb1-41d0-81cf-4864d64f6ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:57.322102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058d2a7-dbb1-41d0-81cf-4864d64f6ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:57.344907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9058d2a7-dbb1-41d0-81cf-4864d64f6ec1 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:09:57.346233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9058d2a7-dbb1-41d0-81cf-4864d64f6ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:57.435207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 983/1990] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:09:57.446114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:57.447618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:57.465667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:57.525706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a02fa9-afc1-4dae-bce9-421ca6c58b8f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:57.530321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d515c-362b-4c42-9331-710ead98917d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:57.531174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d515c-362b-4c42-9331-710ead98917d service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:57.531457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d515c-362b-4c42-9331-710ead98917d service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:57.531759 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d515c-362b-4c42-9331-710ead98917d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:57.545698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d515c-362b-4c42-9331-710ead98917d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:57.549024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d515c-362b-4c42-9331-710ead98917d service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:57.574990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1008/1991] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:09:57.943022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058d2a7-dbb1-41d0-81cf-4864d64f6ec1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:57.945023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9058d2a7-dbb1-41d0-81cf-4864d64f6ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:57.993491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058d2a7-dbb1-41d0-81cf-4864d64f6ec1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:58.001118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:58.002678 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:58.003327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:58.003779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] PATH_INFO: `/v3/projects/6c6fbfe7ffa54f9baa3ffe57030afe04` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:58.006638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c6fbfe7ffa54f9baa3ffe57030afe04)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:58.013590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:58.016647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:58.098031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c6fbfe7ffa54f9baa3ffe57030afe04', 'request_id': 'req-a3b36f93-a9af-4315-aefc-d87f953e6b6d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:58.106766 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c6fbfe7ffa54f9baa3ffe57030afe04', 'request_id': 'req-a3b36f93-a9af-4315-aefc-d87f953e6b6d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:58.112740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c6fbfe7ffa54f9baa3ffe57030afe04', 'request_id': 'req-a3b36f93-a9af-4315-aefc-d87f953e6b6d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:58.132892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] The token cache is being invalidate because project 6c6fbfe7ffa54f9baa3ffe57030afe04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:58.133108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:58.133751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:58.135349 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 984/1992] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:57 2022] DELETE /identity/v3/projects/6c6fbfe7ffa54f9baa3ffe57030afe04 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:09:58.140474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d515c-362b-4c42-9331-710ead98917d service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:58.158622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d515c-362b-4c42-9331-710ead98917d service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:58.252182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:58.253446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:58.253935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:58.254254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] PATH_INFO: `/v3/users/33049581450e47a9aae57961ed5e670a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:58.256359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33049581450e47a9aae57961ed5e670a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:58.258385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:58.322321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33049581450e47a9aae57961ed5e670a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:58.323479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33049581450e47a9aae57961ed5e670a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:58.324111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33049581450e47a9aae57961ed5e670a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:58.336390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33049581450e47a9aae57961ed5e670a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:58.361263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33049581450e47a9aae57961ed5e670a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:58.376899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1009/1993] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:58 2022] DELETE /identity/v3/users/33049581450e47a9aae57961ed5e670a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:59.028734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:59.034485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:59.083567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b36f93-a9af-4315-aefc-d87f953e6b6d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:59.175358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:59.176571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:59.176745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:59.177559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] PATH_INFO: `/v3/projects/1cc690dd898d4d13a74473a7dccb2875` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:59.179831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc690dd898d4d13a74473a7dccb2875)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:59.183366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:59.276989 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc690dd898d4d13a74473a7dccb2875', 'request_id': 'req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:59.299396 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc690dd898d4d13a74473a7dccb2875', 'request_id': 'req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:59.310854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc690dd898d4d13a74473a7dccb2875', 'request_id': 'req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:59.327967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] The token cache is being invalidate because project 1cc690dd898d4d13a74473a7dccb2875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:09:59.328534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:59.329288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:59.331268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 985/1994] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:09:59 2022] DELETE /identity/v3/projects/1cc690dd898d4d13a74473a7dccb2875 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:09:59.336306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:09:59.338813 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:09:59.360245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ecc61-ed30-47be-b67a-14531dfdbdf7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:09:59.404658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:09:59.464563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:09:59.465655 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:09:59.466030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:09:59.466367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] PATH_INFO: `/v3/users/da700bc1bc8d45848933264e31a95904` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:09:59.467872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da700bc1bc8d45848933264e31a95904)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:09:59.470233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:09:59.555615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da700bc1bc8d45848933264e31a95904'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:59.557088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da700bc1bc8d45848933264e31a95904'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:59.557816 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da700bc1bc8d45848933264e31a95904'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:59.571460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da700bc1bc8d45848933264e31a95904'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:59.599894 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3611b4ce-1199-450e-b84e-9d8ffa861db6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da700bc1bc8d45848933264e31a95904'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:09:59.639926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1010/1995] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:09:59 2022] DELETE /identity/v3/users/da700bc1bc8d45848933264e31a95904 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:00.176292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:00.180794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:00.214164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb4c75-0e2e-4575-85e9-8d8ebfc5c750 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:00.243758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:00.326795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:00.328305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:00.328829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:00.329336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] PATH_INFO: `/v3/projects/05798f1e62ef4eb3b8f87a5225e80317` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:00.331654 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05798f1e62ef4eb3b8f87a5225e80317)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:00.336751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:00.435735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05798f1e62ef4eb3b8f87a5225e80317', 'request_id': 'req-a72ff938-bacd-4e5f-8b32-0a75576a6b96'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:00.445142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05798f1e62ef4eb3b8f87a5225e80317', 'request_id': 'req-a72ff938-bacd-4e5f-8b32-0a75576a6b96'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:00.454652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05798f1e62ef4eb3b8f87a5225e80317', 'request_id': 'req-a72ff938-bacd-4e5f-8b32-0a75576a6b96'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:00.477345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] The token cache is being invalidate because project 05798f1e62ef4eb3b8f87a5225e80317 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:00.477758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:00.478779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a72ff938-bacd-4e5f-8b32-0a75576a6b96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:00.485158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 986/1996] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:00 2022] DELETE /identity/v3/projects/05798f1e62ef4eb3b8f87a5225e80317 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:00.487108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82ab8-d00d-4dc8-bb1b-ceba51d26dac None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:00.487601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82ab8-d00d-4dc8-bb1b-ceba51d26dac None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:00.487919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef82ab8-d00d-4dc8-bb1b-ceba51d26dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:00.516952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bef82ab8-d00d-4dc8-bb1b-ceba51d26dac None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:00.519520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bef82ab8-d00d-4dc8-bb1b-ceba51d26dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:00.686753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1011/1997] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:00.698012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5a03-e5c8-4d4d-9ce4-a44df045ee5a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:00.698334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5a03-e5c8-4d4d-9ce4-a44df045ee5a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:00.698616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5a03-e5c8-4d4d-9ce4-a44df045ee5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:00.725999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-597b5a03-e5c8-4d4d-9ce4-a44df045ee5a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:00.727315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-597b5a03-e5c8-4d4d-9ce4-a44df045ee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:00.737290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 987/1998] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:00.742073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82ab8-d00d-4dc8-bb1b-ceba51d26dac None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:00.753935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef82ab8-d00d-4dc8-bb1b-ceba51d26dac None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:00.760215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0147f67-ab04-45e4-b281-327b6af0864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:00.760843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0147f67-ab04-45e4-b281-327b6af0864f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:00.761123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0147f67-ab04-45e4-b281-327b6af0864f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:00.761396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0147f67-ab04-45e4-b281-327b6af0864f admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:00.762423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0147f67-ab04-45e4-b281-327b6af0864f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:10:00.762558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0147f67-ab04-45e4-b281-327b6af0864f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:00.765067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0147f67-ab04-45e4-b281-327b6af0864f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:00.791907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5a03-e5c8-4d4d-9ce4-a44df045ee5a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:00.799398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1012/1999] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:10:00.804847 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5a03-e5c8-4d4d-9ce4-a44df045ee5a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:00.817979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03ea9f-e350-46ef-8af2-fe99e39aabfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:00.818698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03ea9f-e350-46ef-8af2-fe99e39aabfe admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:00.819213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03ea9f-e350-46ef-8af2-fe99e39aabfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:00.819580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d03ea9f-e350-46ef-8af2-fe99e39aabfe admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:00.820542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03ea9f-e350-46ef-8af2-fe99e39aabfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:00.822825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d03ea9f-e350-46ef-8af2-fe99e39aabfe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:00.894049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 988/2000] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:00 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:10:00.900201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0147f67-ab04-45e4-b281-327b6af0864f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:00.921216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0147f67-ab04-45e4-b281-327b6af0864f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:00.981238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:01.035042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40ee56-6902-4513-a34d-9c69dd83f35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.036482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40ee56-6902-4513-a34d-9c69dd83f35e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.036953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40ee56-6902-4513-a34d-9c69dd83f35e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.037433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40ee56-6902-4513-a34d-9c69dd83f35e admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.038280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40ee56-6902-4513-a34d-9c69dd83f35e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.040431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd40ee56-6902-4513-a34d-9c69dd83f35e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.077522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1013/2001] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:00 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:01.078784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03ea9f-e350-46ef-8af2-fe99e39aabfe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.097719 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d03ea9f-e350-46ef-8af2-fe99e39aabfe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.104084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b2939-0c1f-47dd-b5c8-3b61131b25d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.104714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b2939-0c1f-47dd-b5c8-3b61131b25d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.105134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b2939-0c1f-47dd-b5c8-3b61131b25d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.105508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b2939-0c1f-47dd-b5c8-3b61131b25d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.106123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b2939-0c1f-47dd-b5c8-3b61131b25d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.107916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b2939-0c1f-47dd-b5c8-3b61131b25d0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.116591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40ee56-6902-4513-a34d-9c69dd83f35e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.119026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bd40ee56-6902-4513-a34d-9c69dd83f35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:01.122975 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 989/2002] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:01.129306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b2939-0c1f-47dd-b5c8-3b61131b25d0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.137834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40ee56-6902-4513-a34d-9c69dd83f35e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.144556 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.145897 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.146183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.146604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] PATH_INFO: `/v3/users/593a0461ca714cff98ae43eb8833197c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.147969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] RBAC: Authorizing `identity:delete_user(user_id=593a0461ca714cff98ae43eb8833197c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.149882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.157842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b2939-0c1f-47dd-b5c8-3b61131b25d0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.169712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cd2da-55b7-4ae4-afa1-4c9fcb6ec476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.170497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cd2da-55b7-4ae4-afa1-4c9fcb6ec476 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.170778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cd2da-55b7-4ae4-afa1-4c9fcb6ec476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.171051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62cd2da-55b7-4ae4-afa1-4c9fcb6ec476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.171758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cd2da-55b7-4ae4-afa1-4c9fcb6ec476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.174486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62cd2da-55b7-4ae4-afa1-4c9fcb6ec476 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.194386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 990/2003] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:01.199389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cd2da-55b7-4ae4-afa1-4c9fcb6ec476 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.215957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62cd2da-55b7-4ae4-afa1-4c9fcb6ec476 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.220091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '593a0461ca714cff98ae43eb8833197c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:01.220817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '593a0461ca714cff98ae43eb8833197c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:01.221221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41601bfe-453a-4322-84b7-a04ec71bd109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.221439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '593a0461ca714cff98ae43eb8833197c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:01.221931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41601bfe-453a-4322-84b7-a04ec71bd109 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.222213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41601bfe-453a-4322-84b7-a04ec71bd109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.222490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41601bfe-453a-4322-84b7-a04ec71bd109 admin admin] PATH_INFO: `/v3/projects/2c3c0cab80f34f37b510ce8b258edb6b/users/b0091db241b24ee3a6e56abd5cccca5a/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.231779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '593a0461ca714cff98ae43eb8833197c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:01.241399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41601bfe-453a-4322-84b7-a04ec71bd109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c3c0cab80f34f37b510ce8b258edb6b, user_id=b0091db241b24ee3a6e56abd5cccca5a, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.244296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41601bfe-453a-4322-84b7-a04ec71bd109 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.246724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '593a0461ca714cff98ae43eb8833197c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:01.273290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 991/2005] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:01 2022] PUT /identity/v3/projects/2c3c0cab80f34f37b510ce8b258edb6b/users/b0091db241b24ee3a6e56abd5cccca5a/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:01.283546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1014/2005] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:01 2022] DELETE /identity/v3/users/593a0461ca714cff98ae43eb8833197c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:01.286216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edebae9-2bc5-4875-9b1d-f0671c2e29c6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.286504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edebae9-2bc5-4875-9b1d-f0671c2e29c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.286983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edebae9-2bc5-4875-9b1d-f0671c2e29c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.325340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1edebae9-2bc5-4875-9b1d-f0671c2e29c6 None None] MFA Rules not processed for user `b0091db241b24ee3a6e56abd5cccca5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:01.326916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1edebae9-2bc5-4875-9b1d-f0671c2e29c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:01.375987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:01.429538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 992/2006] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:01.433962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.435638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:01.510488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edebae9-2bc5-4875-9b1d-f0671c2e29c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.513226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1edebae9-2bc5-4875-9b1d-f0671c2e29c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:01.529565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4fec3-705a-4680-96ec-ea3d93fc126e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.540031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8839a1-fe47-45d4-aaa4-d34a1cbfe827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.540737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8839a1-fe47-45d4-aaa4-d34a1cbfe827 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.541042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8839a1-fe47-45d4-aaa4-d34a1cbfe827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.541343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8839a1-fe47-45d4-aaa4-d34a1cbfe827 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.541999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edebae9-2bc5-4875-9b1d-f0671c2e29c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.542118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8839a1-fe47-45d4-aaa4-d34a1cbfe827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.544721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8839a1-fe47-45d4-aaa4-d34a1cbfe827 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.558697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.560128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.560128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.560372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] PATH_INFO: `/v3/projects/c0321d32b9014fc0ae63387d3a5032e8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.562917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0321d32b9014fc0ae63387d3a5032e8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.565241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.585793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1015/2007] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:01 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:01.587122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8839a1-fe47-45d4-aaa4-d34a1cbfe827 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.628262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0321d32b9014fc0ae63387d3a5032e8', 'request_id': 'req-982da433-0e2f-4cf6-b70d-76fc17e223d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:01.636188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0321d32b9014fc0ae63387d3a5032e8', 'request_id': 'req-982da433-0e2f-4cf6-b70d-76fc17e223d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:01.642387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:01.643553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0321d32b9014fc0ae63387d3a5032e8', 'request_id': 'req-982da433-0e2f-4cf6-b70d-76fc17e223d6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:01.668072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] The token cache is being invalidate because project c0321d32b9014fc0ae63387d3a5032e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:01.668580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:01.669851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982da433-0e2f-4cf6-b70d-76fc17e223d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:01.672733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 993/2008] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:01 2022] DELETE /identity/v3/projects/c0321d32b9014fc0ae63387d3a5032e8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:01.680410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424d8cf-630e-44fd-b3d1-1f0307dc6e5d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.680898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424d8cf-630e-44fd-b3d1-1f0307dc6e5d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.681395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424d8cf-630e-44fd-b3d1-1f0307dc6e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.687375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8839a1-fe47-45d4-aaa4-d34a1cbfe827 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.711926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6424d8cf-630e-44fd-b3d1-1f0307dc6e5d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:01.713232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6424d8cf-630e-44fd-b3d1-1f0307dc6e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:01.773479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6270d-1f5a-420e-9b6f-ec385dde8d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.774055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6270d-1f5a-420e-9b6f-ec385dde8d6a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.774348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6270d-1f5a-420e-9b6f-ec385dde8d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.774597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6270d-1f5a-420e-9b6f-ec385dde8d6a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.775238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6270d-1f5a-420e-9b6f-ec385dde8d6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.777281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6270d-1f5a-420e-9b6f-ec385dde8d6a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.801202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1016/2009] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:01 2022] POST /identity/v3/users => generated 299 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:10:01.805957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6270d-1f5a-420e-9b6f-ec385dde8d6a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.808367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c1c6270d-1f5a-420e-9b6f-ec385dde8d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:01.811766 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 994/2010] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:01.820738 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d35da-acad-4c6e-9b28-dbc94d968e5c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.821103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d35da-acad-4c6e-9b28-dbc94d968e5c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.821293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d35da-acad-4c6e-9b28-dbc94d968e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.825632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6270d-1f5a-420e-9b6f-ec385dde8d6a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.831111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f37be-59bf-4528-9709-8316639fac97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.831750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f37be-59bf-4528-9709-8316639fac97 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.832066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f37be-59bf-4528-9709-8316639fac97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.832342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f37be-59bf-4528-9709-8316639fac97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.833324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f37be-59bf-4528-9709-8316639fac97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.835378 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f37be-59bf-4528-9709-8316639fac97 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.846771 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-532d35da-acad-4c6e-9b28-dbc94d968e5c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:01.848256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1017/2011] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:01.848459 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-532d35da-acad-4c6e-9b28-dbc94d968e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:01.853408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f37be-59bf-4528-9709-8316639fac97 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.862530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 995/2012] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:01.863687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d35da-acad-4c6e-9b28-dbc94d968e5c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.866928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f37be-59bf-4528-9709-8316639fac97 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.873959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e0fab-795d-4cf0-b854-bfdc512fa5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.874857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e0fab-795d-4cf0-b854-bfdc512fa5c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.875176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e0fab-795d-4cf0-b854-bfdc512fa5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.875463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965e0fab-795d-4cf0-b854-bfdc512fa5c1 admin admin] PATH_INFO: `/v3/projects/e8bfc56a54004846a518d6058e2fbaab/users/623134c872514fce9c76e2e995cf4c2e/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.875632 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d35da-acad-4c6e-9b28-dbc94d968e5c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.881867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbf05e-2604-41df-8cc3-7751f27924f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:01.882505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbf05e-2604-41df-8cc3-7751f27924f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:01.882820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbf05e-2604-41df-8cc3-7751f27924f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:01.883097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbf05e-2604-41df-8cc3-7751f27924f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:01.883760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbf05e-2604-41df-8cc3-7751f27924f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:10:01.884068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbf05e-2604-41df-8cc3-7751f27924f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.886241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbf05e-2604-41df-8cc3-7751f27924f0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.897236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e0fab-795d-4cf0-b854-bfdc512fa5c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8bfc56a54004846a518d6058e2fbaab, user_id=623134c872514fce9c76e2e995cf4c2e, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:01.900157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965e0fab-795d-4cf0-b854-bfdc512fa5c1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:01.902927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 996/2013] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:10:01.909627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbf05e-2604-41df-8cc3-7751f27924f0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.920722 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1018/2014] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:01 2022] PUT /identity/v3/projects/e8bfc56a54004846a518d6058e2fbaab/users/623134c872514fce9c76e2e995cf4c2e/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:01.925172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e0fab-795d-4cf0-b854-bfdc512fa5c1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:01.928693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbf05e-2604-41df-8cc3-7751f27924f0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:01.967715 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:02.002679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965e0fab-795d-4cf0-b854-bfdc512fa5c1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:02.008417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e9049-be1c-4efe-aaa5-4ee4f628ceb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.009052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9049-be1c-4efe-aaa5-4ee4f628ceb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.009325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9049-be1c-4efe-aaa5-4ee4f628ceb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.009581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e9049-be1c-4efe-aaa5-4ee4f628ceb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.010265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e9049-be1c-4efe-aaa5-4ee4f628ceb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.011883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e9049-be1c-4efe-aaa5-4ee4f628ceb6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.018645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd05066-1b7f-4eb6-a14c-8069defbf8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.019268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd05066-1b7f-4eb6-a14c-8069defbf8ed admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.019548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd05066-1b7f-4eb6-a14c-8069defbf8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.019856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd05066-1b7f-4eb6-a14c-8069defbf8ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.020812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd05066-1b7f-4eb6-a14c-8069defbf8ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.022793 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd05066-1b7f-4eb6-a14c-8069defbf8ed admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.023669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1019/2015] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:02.029723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e9049-be1c-4efe-aaa5-4ee4f628ceb6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:02.044270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e9049-be1c-4efe-aaa5-4ee4f628ceb6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:02.051064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85544bf-08c5-4bf9-8b03-19297bf57062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.052622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85544bf-08c5-4bf9-8b03-19297bf57062 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.052911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85544bf-08c5-4bf9-8b03-19297bf57062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.053283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85544bf-08c5-4bf9-8b03-19297bf57062 admin admin] PATH_INFO: `/v3/projects/e8bfc56a54004846a518d6058e2fbaab/users/623134c872514fce9c76e2e995cf4c2e/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.055409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 997/2016] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:01 2022] POST /identity/v3/projects => generated 405 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:02.058478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85544bf-08c5-4bf9-8b03-19297bf57062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8bfc56a54004846a518d6058e2fbaab, user_id=623134c872514fce9c76e2e995cf4c2e, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.059743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd05066-1b7f-4eb6-a14c-8069defbf8ed admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:02.061504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85544bf-08c5-4bf9-8b03-19297bf57062 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.077603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1020/2017] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:02 2022] PUT /identity/v3/projects/e8bfc56a54004846a518d6058e2fbaab/users/623134c872514fce9c76e2e995cf4c2e/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:02.088459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3e22f-acd0-4057-8f1e-c34d9bb24d65 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.088779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3e22f-acd0-4057-8f1e-c34d9bb24d65 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.089058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3e22f-acd0-4057-8f1e-c34d9bb24d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.115048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:02.124046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-94d3e22f-acd0-4057-8f1e-c34d9bb24d65 None None] MFA Rules not processed for user `623134c872514fce9c76e2e995cf4c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:02.125927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-94d3e22f-acd0-4057-8f1e-c34d9bb24d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:02.142880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd05066-1b7f-4eb6-a14c-8069defbf8ed admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:02.205065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1021/2018] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:02.205954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bbe00-64cd-49a0-99bc-4521cc818642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.206581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bbe00-64cd-49a0-99bc-4521cc818642 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.207143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bbe00-64cd-49a0-99bc-4521cc818642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.207143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0bbe00-64cd-49a0-99bc-4521cc818642 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.207839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bbe00-64cd-49a0-99bc-4521cc818642 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.209926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0bbe00-64cd-49a0-99bc-4521cc818642 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.212597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3e22f-acd0-4057-8f1e-c34d9bb24d65 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:02.214043 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-94d3e22f-acd0-4057-8f1e-c34d9bb24d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:02.236115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 998/2019] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:02 2022] POST /identity/v3/users => generated 303 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:10:02.241122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bbe00-64cd-49a0-99bc-4521cc818642 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:02.251276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0bbe00-64cd-49a0-99bc-4521cc818642 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:02.257695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff117ada-497e-4cfa-8b09-5f981fa5341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.258847 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff117ada-497e-4cfa-8b09-5f981fa5341f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.258847 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff117ada-497e-4cfa-8b09-5f981fa5341f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.259044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff117ada-497e-4cfa-8b09-5f981fa5341f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.259541 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff117ada-497e-4cfa-8b09-5f981fa5341f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.261138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff117ada-497e-4cfa-8b09-5f981fa5341f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.276120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 999/2020] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:02.276120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff117ada-497e-4cfa-8b09-5f981fa5341f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:02.276120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ff117ada-497e-4cfa-8b09-5f981fa5341f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:02.287025 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3e22f-acd0-4057-8f1e-c34d9bb24d65 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:02.289152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff117ada-497e-4cfa-8b09-5f981fa5341f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:02.290984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de9ab4-383f-40c2-a0ca-1a7838e918cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.291646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de9ab4-383f-40c2-a0ca-1a7838e918cb service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.291930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de9ab4-383f-40c2-a0ca-1a7838e918cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.292229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de9ab4-383f-40c2-a0ca-1a7838e918cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.294096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-31de9ab4-383f-40c2-a0ca-1a7838e918cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:02.294267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.294735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.295044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.295352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] PATH_INFO: `/v3/users/6ae03636dbf946ab8259f5d9e2848a29` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.296528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ae03636dbf946ab8259f5d9e2848a29)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.298711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.350531 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:02.373283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ae03636dbf946ab8259f5d9e2848a29'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:02.384032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ae03636dbf946ab8259f5d9e2848a29'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:02.384783 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ae03636dbf946ab8259f5d9e2848a29'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:02.386099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ae03636dbf946ab8259f5d9e2848a29'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:02.391948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de9ab4-383f-40c2-a0ca-1a7838e918cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.394313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9465088d-402c-44df-8870-8ddde9d7ee41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ae03636dbf946ab8259f5d9e2848a29'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:02.394592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de9ab4-383f-40c2-a0ca-1a7838e918cb service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.411671 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1022/2021] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:10:02.412991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de9ab4-383f-40c2-a0ca-1a7838e918cb service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:02.431320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de9ab4-383f-40c2-a0ca-1a7838e918cb service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:02.436363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1000/2022] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:02 2022] DELETE /identity/v3/users/6ae03636dbf946ab8259f5d9e2848a29 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:02.439616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76241af2-ced5-4ba1-b7af-aa20e598cfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.439616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76241af2-ced5-4ba1-b7af-aa20e598cfbb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.439616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76241af2-ced5-4ba1-b7af-aa20e598cfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.439616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76241af2-ced5-4ba1-b7af-aa20e598cfbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.439997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76241af2-ced5-4ba1-b7af-aa20e598cfbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.440725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76241af2-ced5-4ba1-b7af-aa20e598cfbb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.443735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02ed27-0430-437e-bf2b-5fd33790ba46 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.444590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02ed27-0430-437e-bf2b-5fd33790ba46 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.445139 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02ed27-0430-437e-bf2b-5fd33790ba46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.461091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1023/2023] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:02.462300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76241af2-ced5-4ba1-b7af-aa20e598cfbb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:02.464690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-76241af2-ced5-4ba1-b7af-aa20e598cfbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:02.476621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6b02ed27-0430-437e-bf2b-5fd33790ba46 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:02.478030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6b02ed27-0430-437e-bf2b-5fd33790ba46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:02.489552 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76241af2-ced5-4ba1-b7af-aa20e598cfbb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:02.552141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:02.579157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8860de-ebfe-4a94-a76c-6a16084fa64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.580403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8860de-ebfe-4a94-a76c-6a16084fa64b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.581616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1001/2024] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:02.581875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8860de-ebfe-4a94-a76c-6a16084fa64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.582214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8860de-ebfe-4a94-a76c-6a16084fa64b admin admin] PATH_INFO: `/v3/projects/b36c217a0377438ab101550adbc2dfd9/users/c563377214c34fef846b9f6446dcfaf7/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.591410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02ed27-0430-437e-bf2b-5fd33790ba46 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:02.599338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6b02ed27-0430-437e-bf2b-5fd33790ba46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:02.608511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8860de-ebfe-4a94-a76c-6a16084fa64b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36c217a0377438ab101550adbc2dfd9, user_id=c563377214c34fef846b9f6446dcfaf7, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.611380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8860de-ebfe-4a94-a76c-6a16084fa64b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.632741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1024/2025] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:02 2022] PUT /identity/v3/projects/b36c217a0377438ab101550adbc2dfd9/users/c563377214c34fef846b9f6446dcfaf7/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:02.643082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd8c76-fe6a-4d22-b08c-f7effe3ef6c1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.643467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd8c76-fe6a-4d22-b08c-f7effe3ef6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.643748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd8c76-fe6a-4d22-b08c-f7effe3ef6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.673952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7dfd8c76-fe6a-4d22-b08c-f7effe3ef6c1 None None] MFA Rules not processed for user `c563377214c34fef846b9f6446dcfaf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:02.675655 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7dfd8c76-fe6a-4d22-b08c-f7effe3ef6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:02.680058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02ed27-0430-437e-bf2b-5fd33790ba46 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:02.744756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91cd66-79d7-4c54-9fab-b6a57ed056ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.745384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91cd66-79d7-4c54-9fab-b6a57ed056ae service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.745706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91cd66-79d7-4c54-9fab-b6a57ed056ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.746045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91cd66-79d7-4c54-9fab-b6a57ed056ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.763084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1025/2026] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:02 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:02.767709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd8c76-fe6a-4d22-b08c-f7effe3ef6c1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:02.781361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:02.851054 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91cd66-79d7-4c54-9fab-b6a57ed056ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.852552 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91cd66-79d7-4c54-9fab-b6a57ed056ae service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.864343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1002/2027] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:10:02.867984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd8c76-fe6a-4d22-b08c-f7effe3ef6c1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:02.874159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a49f5-0ed2-4a7e-9963-37d068da39f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:02.874796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a49f5-0ed2-4a7e-9963-37d068da39f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:02.875076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a49f5-0ed2-4a7e-9963-37d068da39f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:02.875502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a49f5-0ed2-4a7e-9963-37d068da39f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:02.876297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a49f5-0ed2-4a7e-9963-37d068da39f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:02.878672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a49f5-0ed2-4a7e-9963-37d068da39f6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:02.915340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1026/2028] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:02 2022] POST /identity/v3/projects => generated 403 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:02.919616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91cd66-79d7-4c54-9fab-b6a57ed056ae service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:02.930988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91cd66-79d7-4c54-9fab-b6a57ed056ae service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.010437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bbe56-1a99-4423-b550-22a04206dccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.011059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bbe56-1a99-4423-b550-22a04206dccc admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.011338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bbe56-1a99-4423-b550-22a04206dccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.011617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bbe56-1a99-4423-b550-22a04206dccc admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.012466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bbe56-1a99-4423-b550-22a04206dccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.014545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65bbe56-1a99-4423-b550-22a04206dccc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.046639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1003/2029] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:02 2022] POST /identity/v3/users => generated 308 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:03.050855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a49f5-0ed2-4a7e-9963-37d068da39f6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.063699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a49f5-0ed2-4a7e-9963-37d068da39f6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.070719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ea54-7f1e-4f3c-a227-afc5b7de07ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.071850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ea54-7f1e-4f3c-a227-afc5b7de07ec admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.072503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ea54-7f1e-4f3c-a227-afc5b7de07ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.077792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595ea54-7f1e-4f3c-a227-afc5b7de07ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.078501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595ea54-7f1e-4f3c-a227-afc5b7de07ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.080297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595ea54-7f1e-4f3c-a227-afc5b7de07ec admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.104854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1027/2030] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:03.111952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bbe56-1a99-4423-b550-22a04206dccc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.124001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bbe56-1a99-4423-b550-22a04206dccc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.134013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c6699-e147-486b-84aa-5fc88b2282d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.134013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c6699-e147-486b-84aa-5fc88b2282d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.134381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c6699-e147-486b-84aa-5fc88b2282d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.134610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c6699-e147-486b-84aa-5fc88b2282d7 admin admin] PATH_INFO: `/v3/projects/b49b908d9db74060aaac0c82a5e1a026/users/dcfaeb78eac5413f9c7025a18ee4152b/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.153313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c6699-e147-486b-84aa-5fc88b2282d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b49b908d9db74060aaac0c82a5e1a026, user_id=dcfaeb78eac5413f9c7025a18ee4152b, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.156287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c6699-e147-486b-84aa-5fc88b2282d7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.173955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1004/2031] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:03 2022] PUT /identity/v3/projects/b49b908d9db74060aaac0c82a5e1a026/users/dcfaeb78eac5413f9c7025a18ee4152b/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:03.179412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ea54-7f1e-4f3c-a227-afc5b7de07ec admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.194576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595ea54-7f1e-4f3c-a227-afc5b7de07ec admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.233678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:03.246351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c6699-e147-486b-84aa-5fc88b2282d7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.249314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c7c6699-e147-486b-84aa-5fc88b2282d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:03.288254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b2262-af21-4def-8160-03560c186a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.289332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b2262-af21-4def-8160-03560c186a95 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.289867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b2262-af21-4def-8160-03560c186a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.290369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5b2262-af21-4def-8160-03560c186a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.291582 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b2262-af21-4def-8160-03560c186a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.294683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5b2262-af21-4def-8160-03560c186a95 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.309313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1028/2032] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:03.314425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b2262-af21-4def-8160-03560c186a95 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.327811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5b2262-af21-4def-8160-03560c186a95 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.336324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b865b2-765d-42f9-bb61-7ec1763faa7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.337474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b865b2-765d-42f9-bb61-7ec1763faa7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.338029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b865b2-765d-42f9-bb61-7ec1763faa7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.338524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b865b2-765d-42f9-bb61-7ec1763faa7e admin admin] PATH_INFO: `/v3/projects/b49b908d9db74060aaac0c82a5e1a026/users/dcfaeb78eac5413f9c7025a18ee4152b/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.341127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b865b2-765d-42f9-bb61-7ec1763faa7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b49b908d9db74060aaac0c82a5e1a026, user_id=dcfaeb78eac5413f9c7025a18ee4152b, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.343080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c6699-e147-486b-84aa-5fc88b2282d7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.346682 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b865b2-765d-42f9-bb61-7ec1763faa7e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.347703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.348927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.349437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.349882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] PATH_INFO: `/v3/projects/d95cd257ccb745b48422cd4ce8e527c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.351036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] RBAC: Authorizing `identity:delete_project(project_id=d95cd257ccb745b48422cd4ce8e527c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.353254 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.366693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1029/2033] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:03 2022] PUT /identity/v3/projects/b49b908d9db74060aaac0c82a5e1a026/users/dcfaeb78eac5413f9c7025a18ee4152b/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:03.385920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c7607-80ff-4f47-b9c9-d2f152a32db9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.386291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c7607-80ff-4f47-b9c9-d2f152a32db9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.386632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c7607-80ff-4f47-b9c9-d2f152a32db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.399788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:03.433030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd95cd257ccb745b48422cd4ce8e527c5', 'request_id': 'req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:03.434551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bb4c7607-80ff-4f47-b9c9-d2f152a32db9 None None] MFA Rules not processed for user `dcfaeb78eac5413f9c7025a18ee4152b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:03.435799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bb4c7607-80ff-4f47-b9c9-d2f152a32db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:03.440410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd95cd257ccb745b48422cd4ce8e527c5', 'request_id': 'req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:03.447800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd95cd257ccb745b48422cd4ce8e527c5', 'request_id': 'req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:03.463003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] The token cache is being invalidate because project d95cd257ccb745b48422cd4ce8e527c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:03.463363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:03.464097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be1b9e15-6757-4a69-a95e-7ac6fca6ccad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:03.465797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1005/2034] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:03 2022] DELETE /identity/v3/projects/d95cd257ccb745b48422cd4ce8e527c5 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:03.471210 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8d73d-ab6e-4bb7-b256-8b494668dcf9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.471588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8d73d-ab6e-4bb7-b256-8b494668dcf9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.471949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c8d73d-ab6e-4bb7-b256-8b494668dcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.501303 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a8c8d73d-ab6e-4bb7-b256-8b494668dcf9 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:03.502644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a8c8d73d-ab6e-4bb7-b256-8b494668dcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:03.524301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1030/2035] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:03.543388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c7607-80ff-4f47-b9c9-d2f152a32db9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.545560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bb4c7607-80ff-4f47-b9c9-d2f152a32db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:03.592898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1006/2036] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:03.594867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa68052-279f-49ab-89af-7bb2e77b95f0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.595291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa68052-279f-49ab-89af-7bb2e77b95f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.595574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa68052-279f-49ab-89af-7bb2e77b95f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.621462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4fa68052-279f-49ab-89af-7bb2e77b95f0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:03.622142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c7607-80ff-4f47-b9c9-d2f152a32db9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.622860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4fa68052-279f-49ab-89af-7bb2e77b95f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:03.628598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c6a6b-b104-4e06-a927-3ed6d836126f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.629224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c6a6b-b104-4e06-a927-3ed6d836126f service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.629590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c6a6b-b104-4e06-a927-3ed6d836126f service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.629989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1c6a6b-b104-4e06-a927-3ed6d836126f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.633634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1007/2037] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:03.638438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa68052-279f-49ab-89af-7bb2e77b95f0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.641396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c6a6b-b104-4e06-a927-3ed6d836126f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.643840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1c6a6b-b104-4e06-a927-3ed6d836126f service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.649403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa68052-279f-49ab-89af-7bb2e77b95f0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.658193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114b0f2-abee-4b0f-ad24-8a43df985e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.659018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114b0f2-abee-4b0f-ad24-8a43df985e67 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.659470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114b0f2-abee-4b0f-ad24-8a43df985e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.659853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0114b0f2-abee-4b0f-ad24-8a43df985e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.661318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114b0f2-abee-4b0f-ad24-8a43df985e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:10:03.661709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114b0f2-abee-4b0f-ad24-8a43df985e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.663116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1031/2038] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:03 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:10:03.664143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0114b0f2-abee-4b0f-ad24-8a43df985e67 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.689861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1008/2039] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:10:03.695351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c6a6b-b104-4e06-a927-3ed6d836126f service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.726976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1c6a6b-b104-4e06-a927-3ed6d836126f service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.735323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142ad28-d25e-45e0-8dcc-b0705e8191dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.736376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142ad28-d25e-45e0-8dcc-b0705e8191dd admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.736880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142ad28-d25e-45e0-8dcc-b0705e8191dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.737720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142ad28-d25e-45e0-8dcc-b0705e8191dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.739532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142ad28-d25e-45e0-8dcc-b0705e8191dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.744979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142ad28-d25e-45e0-8dcc-b0705e8191dd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.812339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1032/2040] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:03 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:03.813649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114b0f2-abee-4b0f-ad24-8a43df985e67 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.827921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0114b0f2-abee-4b0f-ad24-8a43df985e67 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.854945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142ad28-d25e-45e0-8dcc-b0705e8191dd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.857846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b142ad28-d25e-45e0-8dcc-b0705e8191dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:03.891568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:03.908443 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e0aad-e8b5-4b3d-9c8a-8a598738115d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.909293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e0aad-e8b5-4b3d-9c8a-8a598738115d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.909679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e0aad-e8b5-4b3d-9c8a-8a598738115d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.910055 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e0aad-e8b5-4b3d-9c8a-8a598738115d admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.911039 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e0aad-e8b5-4b3d-9c8a-8a598738115d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.913082 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351e0aad-e8b5-4b3d-9c8a-8a598738115d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.941244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1009/2041] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:03 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:03.946302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e0aad-e8b5-4b3d-9c8a-8a598738115d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:03.957940 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e0aad-e8b5-4b3d-9c8a-8a598738115d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.961969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142ad28-d25e-45e0-8dcc-b0705e8191dd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:03.964736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2764a-71dd-427a-a4bc-ebde049c255e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.965735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2764a-71dd-427a-a4bc-ebde049c255e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.966706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2764a-71dd-427a-a4bc-ebde049c255e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.966903 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2764a-71dd-427a-a4bc-ebde049c255e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.967335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:03.967821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2764a-71dd-427a-a4bc-ebde049c255e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.967984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:03.968250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:03.968699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] PATH_INFO: `/v3/users/c563377214c34fef846b9f6446dcfaf7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:03.969744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c563377214c34fef846b9f6446dcfaf7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:03.970958 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea2764a-71dd-427a-a4bc-ebde049c255e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.971763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:03.997310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1010/2042] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:04.002211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2764a-71dd-427a-a4bc-ebde049c255e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:04.016596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2764a-71dd-427a-a4bc-ebde049c255e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:04.021914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc056-1315-42a4-acd7-520e0cce8d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:04.022581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc056-1315-42a4-acd7-520e0cce8d6a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:04.022865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc056-1315-42a4-acd7-520e0cce8d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:04.023141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4fc056-1315-42a4-acd7-520e0cce8d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:04.024248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc056-1315-42a4-acd7-520e0cce8d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:04.025942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4fc056-1315-42a4-acd7-520e0cce8d6a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:04.037920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c563377214c34fef846b9f6446dcfaf7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:04.041792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1011/2043] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:04.041952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c563377214c34fef846b9f6446dcfaf7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:04.041952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c563377214c34fef846b9f6446dcfaf7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:04.044462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc056-1315-42a4-acd7-520e0cce8d6a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:04.046358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce4fc056-1315-42a4-acd7-520e0cce8d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:04.051095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c563377214c34fef846b9f6446dcfaf7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:04.056714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:04.063840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4fc056-1315-42a4-acd7-520e0cce8d6a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:04.072575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-99dee130-54ef-441e-9a89-f7d49f940b51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c563377214c34fef846b9f6446dcfaf7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:04.075314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8723dff-20f7-4aba-9978-723bf76e24d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:04.077081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723dff-20f7-4aba-9978-723bf76e24d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:04.077557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723dff-20f7-4aba-9978-723bf76e24d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:04.078325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8723dff-20f7-4aba-9978-723bf76e24d9 admin admin] PATH_INFO: `/v3/projects/c9e00c88dc6e45f48ca271ce9ba5b80a/users/b023ac4be4774d22af03d88860a985de/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:04.096308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1033/2044] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:03 2022] DELETE /identity/v3/users/c563377214c34fef846b9f6446dcfaf7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:04.096933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34def7e7-cd32-437c-a620-fbbcd818562c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:04.097220 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34def7e7-cd32-437c-a620-fbbcd818562c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:04.097540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34def7e7-cd32-437c-a620-fbbcd818562c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:04.104322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8723dff-20f7-4aba-9978-723bf76e24d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e00c88dc6e45f48ca271ce9ba5b80a, user_id=b023ac4be4774d22af03d88860a985de, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:04.107096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8723dff-20f7-4aba-9978-723bf76e24d9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:04.122975 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-34def7e7-cd32-437c-a620-fbbcd818562c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:04.124492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-34def7e7-cd32-437c-a620-fbbcd818562c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:04.124898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1012/2045] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:04 2022] PUT /identity/v3/projects/c9e00c88dc6e45f48ca271ce9ba5b80a/users/b023ac4be4774d22af03d88860a985de/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:04.138146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e28ba5-9eed-4ee3-8fde-32b7c063f35a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:04.138424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e28ba5-9eed-4ee3-8fde-32b7c063f35a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:04.138685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e28ba5-9eed-4ee3-8fde-32b7c063f35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:04.170814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-56e28ba5-9eed-4ee3-8fde-32b7c063f35a None None] MFA Rules not processed for user `b023ac4be4774d22af03d88860a985de`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:04.172210 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-56e28ba5-9eed-4ee3-8fde-32b7c063f35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:04.200491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1034/2046] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:04.213714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34def7e7-cd32-437c-a620-fbbcd818562c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:04.215709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-34def7e7-cd32-437c-a620-fbbcd818562c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:04.261717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1013/2047] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:04.273226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:04.277679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e28ba5-9eed-4ee3-8fde-32b7c063f35a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:04.280444 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-56e28ba5-9eed-4ee3-8fde-32b7c063f35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:04.305469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34def7e7-cd32-437c-a620-fbbcd818562c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:04.311503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1221d08-b5ab-47fb-955a-2e7dd764f3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:04.311503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1221d08-b5ab-47fb-955a-2e7dd764f3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:04.311805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1221d08-b5ab-47fb-955a-2e7dd764f3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:04.312265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1221d08-b5ab-47fb-955a-2e7dd764f3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:04.331080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1221d08-b5ab-47fb-955a-2e7dd764f3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:04.333131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1221d08-b5ab-47fb-955a-2e7dd764f3e0 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:04.348831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1035/2048] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:10:04.375872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e28ba5-9eed-4ee3-8fde-32b7c063f35a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:04.384477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb94b-296b-458f-bc4f-efd3dcbe13fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:04.385269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb94b-296b-458f-bc4f-efd3dcbe13fd service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:04.385723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb94b-296b-458f-bc4f-efd3dcbe13fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:04.386124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb94b-296b-458f-bc4f-efd3dcbe13fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:04.404489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bb94b-296b-458f-bc4f-efd3dcbe13fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:04.406944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bb94b-296b-458f-bc4f-efd3dcbe13fd service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:04.423188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1014/2049] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:10:04.816838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1221d08-b5ab-47fb-955a-2e7dd764f3e0 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:04.819921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d1221d08-b5ab-47fb-955a-2e7dd764f3e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:04.844460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1221d08-b5ab-47fb-955a-2e7dd764f3e0 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:04.850512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:04.851139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:04.851379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:04.851648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] PATH_INFO: `/v3/projects/b36c217a0377438ab101550adbc2dfd9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:04.853984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36c217a0377438ab101550adbc2dfd9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:04.854785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:04.926390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36c217a0377438ab101550adbc2dfd9', 'request_id': 'req-09e196c0-b296-436c-b4e4-8c6e0686fc8f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:04.935633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36c217a0377438ab101550adbc2dfd9', 'request_id': 'req-09e196c0-b296-436c-b4e4-8c6e0686fc8f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:04.942826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36c217a0377438ab101550adbc2dfd9', 'request_id': 'req-09e196c0-b296-436c-b4e4-8c6e0686fc8f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:04.955785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] The token cache is being invalidate because project b36c217a0377438ab101550adbc2dfd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:04.956096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:04.956780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:04.958396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1036/2050] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:04 2022] DELETE /identity/v3/projects/b36c217a0377438ab101550adbc2dfd9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:04.962869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb94b-296b-458f-bc4f-efd3dcbe13fd service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:04.975636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb94b-296b-458f-bc4f-efd3dcbe13fd service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:05.053002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:05.053908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:05.054200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:05.054483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] PATH_INFO: `/v3/users/dcfaeb78eac5413f9c7025a18ee4152b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:05.056012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcfaeb78eac5413f9c7025a18ee4152b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:05.057994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:05.082787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:05.132401 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcfaeb78eac5413f9c7025a18ee4152b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:05.140564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcfaeb78eac5413f9c7025a18ee4152b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:05.141240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcfaeb78eac5413f9c7025a18ee4152b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:05.142023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcfaeb78eac5413f9c7025a18ee4152b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:05.151491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-72fa6884-7c25-4a63-aa26-0b4b08df39c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcfaeb78eac5413f9c7025a18ee4152b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:05.172119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1015/2051] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:04 2022] DELETE /identity/v3/users/dcfaeb78eac5413f9c7025a18ee4152b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:05.586913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:05.591431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:05.616953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e196c0-b296-436c-b4e4-8c6e0686fc8f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:05.716188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:05.716928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:05.717225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:05.718613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] PATH_INFO: `/v3/projects/b49b908d9db74060aaac0c82a5e1a026` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:05.720078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b49b908d9db74060aaac0c82a5e1a026)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:05.722118 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:05.794533 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b49b908d9db74060aaac0c82a5e1a026', 'request_id': 'req-06da6bc0-6c8f-44b7-9779-4078f2c994e2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:05.804435 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b49b908d9db74060aaac0c82a5e1a026', 'request_id': 'req-06da6bc0-6c8f-44b7-9779-4078f2c994e2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:05.811698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:05.814466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b49b908d9db74060aaac0c82a5e1a026', 'request_id': 'req-06da6bc0-6c8f-44b7-9779-4078f2c994e2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:05.842755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] The token cache is being invalidate because project b49b908d9db74060aaac0c82a5e1a026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:05.843685 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:05.844003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-06da6bc0-6c8f-44b7-9779-4078f2c994e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:05.846785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1037/2052] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:05 2022] DELETE /identity/v3/projects/b49b908d9db74060aaac0c82a5e1a026 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:05.852050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdc6d2-c520-4e6a-bd13-eb31e31930c6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:05.852050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdc6d2-c520-4e6a-bd13-eb31e31930c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:05.852050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefdc6d2-c520-4e6a-bd13-eb31e31930c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:05.883087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fefdc6d2-c520-4e6a-bd13-eb31e31930c6 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:05.884389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fefdc6d2-c520-4e6a-bd13-eb31e31930c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:05.993601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1016/2053] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:06.002232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168d081-9840-4115-a6e3-4f21feec49b0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.002561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168d081-9840-4115-a6e3-4f21feec49b0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.002779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168d081-9840-4115-a6e3-4f21feec49b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.029516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d168d081-9840-4115-a6e3-4f21feec49b0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:06.030809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d168d081-9840-4115-a6e3-4f21feec49b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:06.040828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1038/2054] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:06.046495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdc6d2-c520-4e6a-bd13-eb31e31930c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:06.057647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefdc6d2-c520-4e6a-bd13-eb31e31930c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:06.063324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdbec5-215b-4846-8c88-1d28f8e74384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:06.063966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdbec5-215b-4846-8c88-1d28f8e74384 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.064250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdbec5-215b-4846-8c88-1d28f8e74384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.064523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdbec5-215b-4846-8c88-1d28f8e74384 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.065210 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdbec5-215b-4846-8c88-1d28f8e74384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:10:06.065516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdbec5-215b-4846-8c88-1d28f8e74384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:06.067283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fdbec5-215b-4846-8c88-1d28f8e74384 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:06.084934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1017/2055] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:10:06.086760 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168d081-9840-4115-a6e3-4f21feec49b0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:06.099215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168d081-9840-4115-a6e3-4f21feec49b0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:06.104577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ad998-8eed-4225-941e-feba0824d097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:06.105201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ad998-8eed-4225-941e-feba0824d097 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.105502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ad998-8eed-4225-941e-feba0824d097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.105940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651ad998-8eed-4225-941e-feba0824d097 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.106471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ad998-8eed-4225-941e-feba0824d097 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:06.108593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651ad998-8eed-4225-941e-feba0824d097 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:06.144513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1039/2056] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:06 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:06.150383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdbec5-215b-4846-8c88-1d28f8e74384 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:06.167191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdbec5-215b-4846-8c88-1d28f8e74384 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:06.249854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e601e-ff4e-4c8c-b68f-db8b584484f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:06.250539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e601e-ff4e-4c8c-b68f-db8b584484f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.250765 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e601e-ff4e-4c8c-b68f-db8b584484f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.251042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e601e-ff4e-4c8c-b68f-db8b584484f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.251688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e601e-ff4e-4c8c-b68f-db8b584484f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:06.253947 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e601e-ff4e-4c8c-b68f-db8b584484f2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:06.292343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1018/2057] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:06 2022] POST /identity/v3/users => generated 283 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:06.302488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ad998-8eed-4225-941e-feba0824d097 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:06.308723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651ad998-8eed-4225-941e-feba0824d097 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:06.315373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92132cc4-874b-45cd-8506-985a5976adbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:06.316167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92132cc4-874b-45cd-8506-985a5976adbb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.316575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92132cc4-874b-45cd-8506-985a5976adbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.316875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92132cc4-874b-45cd-8506-985a5976adbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.317833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92132cc4-874b-45cd-8506-985a5976adbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:06.319738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92132cc4-874b-45cd-8506-985a5976adbb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:06.335773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1040/2058] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:06.338284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e601e-ff4e-4c8c-b68f-db8b584484f2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:06.357543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e601e-ff4e-4c8c-b68f-db8b584484f2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:06.365259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734239d-a554-423a-9a69-08ff15bb7885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:06.366642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734239d-a554-423a-9a69-08ff15bb7885 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.367029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734239d-a554-423a-9a69-08ff15bb7885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.367320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734239d-a554-423a-9a69-08ff15bb7885 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.368025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734239d-a554-423a-9a69-08ff15bb7885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:06.369702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734239d-a554-423a-9a69-08ff15bb7885 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:06.372641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:06.386574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1019/2059] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:06.392430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92132cc4-874b-45cd-8506-985a5976adbb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:06.407000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92132cc4-874b-45cd-8506-985a5976adbb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:06.414256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401b6bb-8c58-41eb-b751-8f394dae2c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:06.415185 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401b6bb-8c58-41eb-b751-8f394dae2c63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.415581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401b6bb-8c58-41eb-b751-8f394dae2c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.415918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2401b6bb-8c58-41eb-b751-8f394dae2c63 admin admin] PATH_INFO: `/v3/projects/4853f68dff6c408c998a6e3e386c9915/users/0e16d1c4d13c4f1dbbb8e47173fa84c8/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.433695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401b6bb-8c58-41eb-b751-8f394dae2c63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4853f68dff6c408c998a6e3e386c9915, user_id=0e16d1c4d13c4f1dbbb8e47173fa84c8, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:06.436909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2401b6bb-8c58-41eb-b751-8f394dae2c63 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:06.454192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1041/2060] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:06 2022] PUT /identity/v3/projects/4853f68dff6c408c998a6e3e386c9915/users/0e16d1c4d13c4f1dbbb8e47173fa84c8/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:06.467344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ca57-53e7-4098-bd4a-01b512e9910f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.467842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ca57-53e7-4098-bd4a-01b512e9910f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.468215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381ca57-53e7-4098-bd4a-01b512e9910f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.512397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7381ca57-53e7-4098-bd4a-01b512e9910f None None] MFA Rules not processed for user `0e16d1c4d13c4f1dbbb8e47173fa84c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:06.514304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7381ca57-53e7-4098-bd4a-01b512e9910f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:06.606316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1020/2061] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:06.611007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401b6bb-8c58-41eb-b751-8f394dae2c63 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:06.699183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2401b6bb-8c58-41eb-b751-8f394dae2c63 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:06.704068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1693f-d56d-4cb3-bc2d-68697b48d151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:06.704680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1693f-d56d-4cb3-bc2d-68697b48d151 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.704960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1693f-d56d-4cb3-bc2d-68697b48d151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.705234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1693f-d56d-4cb3-bc2d-68697b48d151 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.705900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1693f-d56d-4cb3-bc2d-68697b48d151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:06.708917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1693f-d56d-4cb3-bc2d-68697b48d151 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:06.739360 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1042/2062] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:06 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:06.744771 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381ca57-53e7-4098-bd4a-01b512e9910f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:06.757784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381ca57-53e7-4098-bd4a-01b512e9910f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:06.834512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686e31f-b14e-4f36-8a45-98b06209b68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:06.835277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686e31f-b14e-4f36-8a45-98b06209b68c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.835565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686e31f-b14e-4f36-8a45-98b06209b68c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.835845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686e31f-b14e-4f36-8a45-98b06209b68c admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.836528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686e31f-b14e-4f36-8a45-98b06209b68c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:06.838537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686e31f-b14e-4f36-8a45-98b06209b68c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:06.874680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1021/2063] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:06 2022] POST /identity/v3/users => generated 289 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:10:06.881197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1693f-d56d-4cb3-bc2d-68697b48d151 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:06.894280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1693f-d56d-4cb3-bc2d-68697b48d151 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:06.907750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0737f15-ba80-43ae-82ee-1532fc167510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:06.908623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0737f15-ba80-43ae-82ee-1532fc167510 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.909012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0737f15-ba80-43ae-82ee-1532fc167510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.909425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0737f15-ba80-43ae-82ee-1532fc167510 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.910543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0737f15-ba80-43ae-82ee-1532fc167510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:06.912570 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0737f15-ba80-43ae-82ee-1532fc167510 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:06.931746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1043/2064] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:06.933933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686e31f-b14e-4f36-8a45-98b06209b68c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:06.951758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686e31f-b14e-4f36-8a45-98b06209b68c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:06.959291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb07b4-4032-46ab-8dcb-97e7beee1740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:06.960458 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb07b4-4032-46ab-8dcb-97e7beee1740 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:06.960957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb07b4-4032-46ab-8dcb-97e7beee1740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:06.961454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb07b4-4032-46ab-8dcb-97e7beee1740 admin admin] PATH_INFO: `/v3/projects/e0c55189ad67442fa2b8b15157567a11/users/7a5b279424a3444ea3d0dcdc3d9f556a/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:06.999566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb07b4-4032-46ab-8dcb-97e7beee1740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0c55189ad67442fa2b8b15157567a11, user_id=7a5b279424a3444ea3d0dcdc3d9f556a, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:07.003600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb07b4-4032-46ab-8dcb-97e7beee1740 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:07.032074 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1022/2065] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:06 2022] PUT /identity/v3/projects/e0c55189ad67442fa2b8b15157567a11/users/7a5b279424a3444ea3d0dcdc3d9f556a/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:07.038610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0737f15-ba80-43ae-82ee-1532fc167510 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:07.053850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0737f15-ba80-43ae-82ee-1532fc167510 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:07.140530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5245f0b-26d1-4cea-aae5-018feed4262f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:07.141158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5245f0b-26d1-4cea-aae5-018feed4262f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:07.141474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5245f0b-26d1-4cea-aae5-018feed4262f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:07.141863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5245f0b-26d1-4cea-aae5-018feed4262f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:07.142475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5245f0b-26d1-4cea-aae5-018feed4262f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:07.144974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5245f0b-26d1-4cea-aae5-018feed4262f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:07.149051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:07.168587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1044/2066] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:07.169820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb07b4-4032-46ab-8dcb-97e7beee1740 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:07.183350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb07b4-4032-46ab-8dcb-97e7beee1740 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:07.190245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9bc25-266c-45ae-8296-27d5126ee23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:07.191005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9bc25-266c-45ae-8296-27d5126ee23d admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:07.191353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9bc25-266c-45ae-8296-27d5126ee23d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:07.191686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe9bc25-266c-45ae-8296-27d5126ee23d admin admin] PATH_INFO: `/v3/projects/e0c55189ad67442fa2b8b15157567a11/users/7a5b279424a3444ea3d0dcdc3d9f556a/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:07.194518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9bc25-266c-45ae-8296-27d5126ee23d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0c55189ad67442fa2b8b15157567a11, user_id=7a5b279424a3444ea3d0dcdc3d9f556a, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:07.197199 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe9bc25-266c-45ae-8296-27d5126ee23d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:07.212031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1023/2067] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:07 2022] PUT /identity/v3/projects/e0c55189ad67442fa2b8b15157567a11/users/7a5b279424a3444ea3d0dcdc3d9f556a/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:07.225266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaefb1-e033-4e36-ab7c-b667ab92ac1c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:07.225266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaefb1-e033-4e36-ab7c-b667ab92ac1c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:07.225853 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaefb1-e033-4e36-ab7c-b667ab92ac1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:07.270808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4acaefb1-e033-4e36-ab7c-b667ab92ac1c None None] MFA Rules not processed for user `7a5b279424a3444ea3d0dcdc3d9f556a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:07.271941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4acaefb1-e033-4e36-ab7c-b667ab92ac1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:07.385702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1045/2068] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:07.398211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9bc25-266c-45ae-8296-27d5126ee23d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:07.402372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8fe9bc25-266c-45ae-8296-27d5126ee23d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:07.474299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:07.492422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe9bc25-266c-45ae-8296-27d5126ee23d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:07.496456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c88a7d-f0ae-4386-981b-38d54bb96ccb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:07.497082 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88a7d-f0ae-4386-981b-38d54bb96ccb service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:07.497358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88a7d-f0ae-4386-981b-38d54bb96ccb service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:07.497640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c88a7d-f0ae-4386-981b-38d54bb96ccb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:07.516283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c88a7d-f0ae-4386-981b-38d54bb96ccb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:07.518153 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c88a7d-f0ae-4386-981b-38d54bb96ccb service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:07.531284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1024/2069] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:07.632133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acaefb1-e033-4e36-ab7c-b667ab92ac1c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:07.634481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4acaefb1-e033-4e36-ab7c-b667ab92ac1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:07.648505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acaefb1-e033-4e36-ab7c-b667ab92ac1c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:07.661790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2ca65-012d-4b3a-b3d8-d16f98c70189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:07.662851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ca65-012d-4b3a-b3d8-d16f98c70189 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:07.663292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ca65-012d-4b3a-b3d8-d16f98c70189 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:07.663615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2ca65-012d-4b3a-b3d8-d16f98c70189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:07.743735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2ca65-012d-4b3a-b3d8-d16f98c70189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:07.746377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2ca65-012d-4b3a-b3d8-d16f98c70189 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:07.761494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1046/2070] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:13.224788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c88a7d-f0ae-4386-981b-38d54bb96ccb service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:13.319641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c88a7d-f0ae-4386-981b-38d54bb96ccb service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:13.325859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:13.326528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:13.326994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:13.327280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] PATH_INFO: `/v3/users/0e16d1c4d13c4f1dbbb8e47173fa84c8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:13.328532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e16d1c4d13c4f1dbbb8e47173fa84c8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:13.330563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:13.389804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e16d1c4d13c4f1dbbb8e47173fa84c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:13.399406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e16d1c4d13c4f1dbbb8e47173fa84c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:13.400052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e16d1c4d13c4f1dbbb8e47173fa84c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:13.401073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e16d1c4d13c4f1dbbb8e47173fa84c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:13.409547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e16d1c4d13c4f1dbbb8e47173fa84c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:13.437231 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1025/2071] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:13 2022] DELETE /identity/v3/users/0e16d1c4d13c4f1dbbb8e47173fa84c8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:13.446043 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f27726-95b9-42d0-bf6a-16e686221024 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:13.446616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f27726-95b9-42d0-bf6a-16e686221024 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:13.447070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f27726-95b9-42d0-bf6a-16e686221024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:13.476428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-38f27726-95b9-42d0-bf6a-16e686221024 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:13.477680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-38f27726-95b9-42d0-bf6a-16e686221024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:13.580890 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1047/2072] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:13.594607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:13.597899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:13.709865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444c6c39-7e74-4857-bf3d-72815b76b46d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:13.715267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3608d83-664a-48b5-93f8-0892c1a50deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:13.715978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3608d83-664a-48b5-93f8-0892c1a50deb service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:13.716345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3608d83-664a-48b5-93f8-0892c1a50deb service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:13.716711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3608d83-664a-48b5-93f8-0892c1a50deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:13.732377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3608d83-664a-48b5-93f8-0892c1a50deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:13.734622 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3608d83-664a-48b5-93f8-0892c1a50deb service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:13.763108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1026/2073] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:14.114752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f27726-95b9-42d0-bf6a-16e686221024 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:14.117114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-38f27726-95b9-42d0-bf6a-16e686221024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:14.140366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f27726-95b9-42d0-bf6a-16e686221024 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:14.151774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:14.153005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:14.154181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:14.156558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] PATH_INFO: `/v3/projects/4853f68dff6c408c998a6e3e386c9915` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:14.159063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4853f68dff6c408c998a6e3e386c9915)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:14.163264 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:14.197269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:14.231038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4853f68dff6c408c998a6e3e386c9915', 'request_id': 'req-8fea40e6-a355-4134-9f1b-70d0f297e8b0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:14.240115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4853f68dff6c408c998a6e3e386c9915', 'request_id': 'req-8fea40e6-a355-4134-9f1b-70d0f297e8b0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:14.247206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4853f68dff6c408c998a6e3e386c9915', 'request_id': 'req-8fea40e6-a355-4134-9f1b-70d0f297e8b0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:14.260398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] The token cache is being invalidate because project 4853f68dff6c408c998a6e3e386c9915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:14.260719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:14.261402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:14.262948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1048/2074] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:14 2022] DELETE /identity/v3/projects/4853f68dff6c408c998a6e3e386c9915 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:14.267321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3608d83-664a-48b5-93f8-0892c1a50deb service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:14.274711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:14.280259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3608d83-664a-48b5-93f8-0892c1a50deb service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:14.349057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:14.349698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:14.350016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:14.350293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] PATH_INFO: `/v3/users/7a5b279424a3444ea3d0dcdc3d9f556a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:14.351531 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5b279424a3444ea3d0dcdc3d9f556a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:14.353523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:14.419529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5b279424a3444ea3d0dcdc3d9f556a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:14.432111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5b279424a3444ea3d0dcdc3d9f556a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:14.432891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5b279424a3444ea3d0dcdc3d9f556a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:14.433664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5b279424a3444ea3d0dcdc3d9f556a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:14.442576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5b279424a3444ea3d0dcdc3d9f556a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:14.466350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1027/2075] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:14 2022] DELETE /identity/v3/users/7a5b279424a3444ea3d0dcdc3d9f556a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:14.784941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:14.788139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:14.817213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:14.827861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:14.922045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea40e6-a355-4134-9f1b-70d0f297e8b0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:14.926880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4d90d-1960-45d3-a055-03f86fc241fa admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:14.938468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:14.939281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:14.939671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:14.940070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] PATH_INFO: `/v3/projects/e0c55189ad67442fa2b8b15157567a11` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:14.941461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0c55189ad67442fa2b8b15157567a11)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:14.943724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:14.980613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:15.010569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0c55189ad67442fa2b8b15157567a11', 'request_id': 'req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:15.013150 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db323b6e-1885-4d01-94e7-2aa0f416a070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:15.013833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323b6e-1885-4d01-94e7-2aa0f416a070 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.014123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323b6e-1885-4d01-94e7-2aa0f416a070 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.014489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323b6e-1885-4d01-94e7-2aa0f416a070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.016581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-db323b6e-1885-4d01-94e7-2aa0f416a070 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:15.023431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0c55189ad67442fa2b8b15157567a11', 'request_id': 'req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:15.033757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0c55189ad67442fa2b8b15157567a11', 'request_id': 'req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:15.055245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] The token cache is being invalidate because project e0c55189ad67442fa2b8b15157567a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:15.055547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:15.056293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5e7b566a-3639-4a7e-b7ac-b5008c2d5169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:15.057893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1028/2076] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:14 2022] DELETE /identity/v3/projects/e0c55189ad67442fa2b8b15157567a11 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:15.064197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27179c63-a9ba-48a2-b622-b2f1463d8a98 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.064761 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27179c63-a9ba-48a2-b622-b2f1463d8a98 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.065275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27179c63-a9ba-48a2-b622-b2f1463d8a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.102132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-27179c63-a9ba-48a2-b622-b2f1463d8a98 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:15.103481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db323b6e-1885-4d01-94e7-2aa0f416a070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:15.104371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-27179c63-a9ba-48a2-b622-b2f1463d8a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:15.104940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db323b6e-1885-4d01-94e7-2aa0f416a070 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:15.116476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1049/2077] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:15.210345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1029/2078] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:15.221212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7353083-788a-462d-b468-900a918fcfaf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.222011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7353083-788a-462d-b468-900a918fcfaf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.222593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7353083-788a-462d-b468-900a918fcfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.257722 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d7353083-788a-462d-b468-900a918fcfaf None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:15.259662 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d7353083-788a-462d-b468-900a918fcfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:15.270087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1050/2079] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:15.274880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27179c63-a9ba-48a2-b622-b2f1463d8a98 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:15.288302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27179c63-a9ba-48a2-b622-b2f1463d8a98 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:15.295617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b024c-2564-4180-bae7-61b54ab9aa61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:15.296493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b024c-2564-4180-bae7-61b54ab9aa61 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.296852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b024c-2564-4180-bae7-61b54ab9aa61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.297237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602b024c-2564-4180-bae7-61b54ab9aa61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.298080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b024c-2564-4180-bae7-61b54ab9aa61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:10:15.298432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b024c-2564-4180-bae7-61b54ab9aa61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:15.300512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602b024c-2564-4180-bae7-61b54ab9aa61 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:15.316539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1030/2080] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:10:15.324231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7353083-788a-462d-b468-900a918fcfaf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:15.330763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:15.336366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7353083-788a-462d-b468-900a918fcfaf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:15.342950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df7a91-b1b0-470c-9005-951f08df8dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:15.344078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df7a91-b1b0-470c-9005-951f08df8dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.344529 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df7a91-b1b0-470c-9005-951f08df8dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.344849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df7a91-b1b0-470c-9005-951f08df8dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.345836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df7a91-b1b0-470c-9005-951f08df8dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:15.349834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df7a91-b1b0-470c-9005-951f08df8dc0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:15.388115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1051/2081] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:15 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:15.392550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b024c-2564-4180-bae7-61b54ab9aa61 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:15.404651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602b024c-2564-4180-bae7-61b54ab9aa61 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:15.514606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da13e1-087f-4809-8394-92bb7352aaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:15.515395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da13e1-087f-4809-8394-92bb7352aaea admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.515569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da13e1-087f-4809-8394-92bb7352aaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.515847 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da13e1-087f-4809-8394-92bb7352aaea admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.517080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da13e1-087f-4809-8394-92bb7352aaea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:15.519315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da13e1-087f-4809-8394-92bb7352aaea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:15.550079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1031/2082] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:15 2022] POST /identity/v3/users => generated 292 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:10:15.557545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df7a91-b1b0-470c-9005-951f08df8dc0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:15.578428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df7a91-b1b0-470c-9005-951f08df8dc0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:15.585046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00b7ad-2304-4cda-8382-f84a45331eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:15.585677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00b7ad-2304-4cda-8382-f84a45331eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.585990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00b7ad-2304-4cda-8382-f84a45331eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.586263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c00b7ad-2304-4cda-8382-f84a45331eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.586922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00b7ad-2304-4cda-8382-f84a45331eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:15.589656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c00b7ad-2304-4cda-8382-f84a45331eb2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:15.603919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1052/2083] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:15.608736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da13e1-087f-4809-8394-92bb7352aaea admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:15.620277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da13e1-087f-4809-8394-92bb7352aaea admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:15.625882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b0313-6191-4820-a397-55474fdc492f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:15.626518 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b0313-6191-4820-a397-55474fdc492f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.626800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b0313-6191-4820-a397-55474fdc492f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.627079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b0313-6191-4820-a397-55474fdc492f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.627743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b0313-6191-4820-a397-55474fdc492f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:15.630501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b0313-6191-4820-a397-55474fdc492f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:15.642797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1032/2084] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:15.648144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00b7ad-2304-4cda-8382-f84a45331eb2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:15.659140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c00b7ad-2304-4cda-8382-f84a45331eb2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:15.665595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d290da-d6f2-48ca-baec-54ea770f1e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:15.666339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d290da-d6f2-48ca-baec-54ea770f1e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.666623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d290da-d6f2-48ca-baec-54ea770f1e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.666892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d290da-d6f2-48ca-baec-54ea770f1e63 admin admin] PATH_INFO: `/v3/projects/df9f16a631fc4268b3ad70bc488ef7f5/users/c00d5055034d465f90a1cde955a86c86/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.685818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d290da-d6f2-48ca-baec-54ea770f1e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df9f16a631fc4268b3ad70bc488ef7f5, user_id=c00d5055034d465f90a1cde955a86c86, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:15.688653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d290da-d6f2-48ca-baec-54ea770f1e63 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:15.703796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1053/2085] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:15 2022] PUT /identity/v3/projects/df9f16a631fc4268b3ad70bc488ef7f5/users/c00d5055034d465f90a1cde955a86c86/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:15.715252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0eec81-1020-4aa2-9822-c2932a038819 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.715532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0eec81-1020-4aa2-9822-c2932a038819 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.715801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0eec81-1020-4aa2-9822-c2932a038819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.749179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0b0eec81-1020-4aa2-9822-c2932a038819 None None] MFA Rules not processed for user `c00d5055034d465f90a1cde955a86c86`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:15.750682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0b0eec81-1020-4aa2-9822-c2932a038819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:15.760748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:15.834537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1033/2086] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:15.839253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d290da-d6f2-48ca-baec-54ea770f1e63 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:15.911683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d290da-d6f2-48ca-baec-54ea770f1e63 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:15.916769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8f59a-2cd4-47cf-9c27-998a11280e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:15.917378 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8f59a-2cd4-47cf-9c27-998a11280e2e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:15.917656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8f59a-2cd4-47cf-9c27-998a11280e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:15.917960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8f59a-2cd4-47cf-9c27-998a11280e2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:15.918600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8f59a-2cd4-47cf-9c27-998a11280e2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:15.920515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad8f59a-2cd4-47cf-9c27-998a11280e2e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:15.951811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1054/2087] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:15 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:10:15.956029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0eec81-1020-4aa2-9822-c2932a038819 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:15.967595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0eec81-1020-4aa2-9822-c2932a038819 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:16.039877 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af83d77-1e6f-4ab8-b5a0-da39f6e5c0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:16.040580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af83d77-1e6f-4ab8-b5a0-da39f6e5c0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:16.040973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af83d77-1e6f-4ab8-b5a0-da39f6e5c0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:16.041333 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af83d77-1e6f-4ab8-b5a0-da39f6e5c0c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:16.042028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af83d77-1e6f-4ab8-b5a0-da39f6e5c0c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:16.044544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af83d77-1e6f-4ab8-b5a0-da39f6e5c0c5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:16.077902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1034/2088] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:15 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:16.081730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8f59a-2cd4-47cf-9c27-998a11280e2e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:16.093746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8f59a-2cd4-47cf-9c27-998a11280e2e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:16.099848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9df61-85ae-44f2-9422-e4c8e7ddea9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:16.100762 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9df61-85ae-44f2-9422-e4c8e7ddea9d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:16.101096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9df61-85ae-44f2-9422-e4c8e7ddea9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:16.101382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9df61-85ae-44f2-9422-e4c8e7ddea9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:16.102289 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9df61-85ae-44f2-9422-e4c8e7ddea9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:16.104115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9df61-85ae-44f2-9422-e4c8e7ddea9d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:16.116958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1055/2089] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:16.121481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af83d77-1e6f-4ab8-b5a0-da39f6e5c0c5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:16.133680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af83d77-1e6f-4ab8-b5a0-da39f6e5c0c5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:16.140838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf8a16-415f-41d7-898f-51728fa65652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:16.141874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf8a16-415f-41d7-898f-51728fa65652 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:16.142288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf8a16-415f-41d7-898f-51728fa65652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:16.142613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf8a16-415f-41d7-898f-51728fa65652 admin admin] PATH_INFO: `/v3/projects/f5c89f5623c14f9296bc340f0ea0299a/users/d4960670fe3d4a88a6d8a163428c437b/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:16.170477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf8a16-415f-41d7-898f-51728fa65652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c89f5623c14f9296bc340f0ea0299a, user_id=d4960670fe3d4a88a6d8a163428c437b, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:16.173256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf8a16-415f-41d7-898f-51728fa65652 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:16.193209 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1035/2090] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:16 2022] PUT /identity/v3/projects/f5c89f5623c14f9296bc340f0ea0299a/users/d4960670fe3d4a88a6d8a163428c437b/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:16.197287 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9df61-85ae-44f2-9422-e4c8e7ddea9d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:16.207892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9df61-85ae-44f2-9422-e4c8e7ddea9d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:16.290223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741eb33a-033c-43d9-94df-1f36bdfb41dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:16.290993 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741eb33a-033c-43d9-94df-1f36bdfb41dc admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:16.291183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741eb33a-033c-43d9-94df-1f36bdfb41dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:16.291396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741eb33a-033c-43d9-94df-1f36bdfb41dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:16.292087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741eb33a-033c-43d9-94df-1f36bdfb41dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:16.293736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741eb33a-033c-43d9-94df-1f36bdfb41dc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:16.305339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1056/2091] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:16.310476 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf8a16-415f-41d7-898f-51728fa65652 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:16.331397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf8a16-415f-41d7-898f-51728fa65652 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:16.338680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfc6a8-0715-442d-b4e3-213cc6bceddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:16.339548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfc6a8-0715-442d-b4e3-213cc6bceddb admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:16.339971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfc6a8-0715-442d-b4e3-213cc6bceddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:16.340439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfc6a8-0715-442d-b4e3-213cc6bceddb admin admin] PATH_INFO: `/v3/projects/f5c89f5623c14f9296bc340f0ea0299a/users/d4960670fe3d4a88a6d8a163428c437b/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:16.343603 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfc6a8-0715-442d-b4e3-213cc6bceddb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c89f5623c14f9296bc340f0ea0299a, user_id=d4960670fe3d4a88a6d8a163428c437b, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:16.346243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfc6a8-0715-442d-b4e3-213cc6bceddb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:16.358366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1036/2092] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:16 2022] PUT /identity/v3/projects/f5c89f5623c14f9296bc340f0ea0299a/users/d4960670fe3d4a88a6d8a163428c437b/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:16.370213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fb762-8d98-45c3-b5ac-35d32a040480 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:16.370566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fb762-8d98-45c3-b5ac-35d32a040480 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:16.370785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fb762-8d98-45c3-b5ac-35d32a040480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:16.404512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a59fb762-8d98-45c3-b5ac-35d32a040480 None None] MFA Rules not processed for user `d4960670fe3d4a88a6d8a163428c437b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:16.405749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a59fb762-8d98-45c3-b5ac-35d32a040480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:16.413703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:16.495517 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1057/2093] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:16.511639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfc6a8-0715-442d-b4e3-213cc6bceddb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:16.513720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-efdfc6a8-0715-442d-b4e3-213cc6bceddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:16.607379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfc6a8-0715-442d-b4e3-213cc6bceddb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:16.607379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02baa2-21ae-4154-9968-6e4688c93934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:16.607379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02baa2-21ae-4154-9968-6e4688c93934 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:16.607379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02baa2-21ae-4154-9968-6e4688c93934 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:16.608019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02baa2-21ae-4154-9968-6e4688c93934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:16.615542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02baa2-21ae-4154-9968-6e4688c93934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:16.617117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02baa2-21ae-4154-9968-6e4688c93934 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:16.628754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1037/2094] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:16.763988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fb762-8d98-45c3-b5ac-35d32a040480 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:16.782131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fb762-8d98-45c3-b5ac-35d32a040480 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:16.789425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33662dc7-9831-4e73-9f45-f1794284d98a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:16.790512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33662dc7-9831-4e73-9f45-f1794284d98a service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:16.791020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33662dc7-9831-4e73-9f45-f1794284d98a service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:16.791513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33662dc7-9831-4e73-9f45-f1794284d98a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:16.892459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33662dc7-9831-4e73-9f45-f1794284d98a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:16.895742 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33662dc7-9831-4e73-9f45-f1794284d98a service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:16.907934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1058/2095] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:26.623298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:26.902551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=54 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:32.313411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02baa2-21ae-4154-9968-6e4688c93934 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:32.376873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02baa2-21ae-4154-9968-6e4688c93934 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:32.382032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:32.382651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:32.382985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:32.383263 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] PATH_INFO: `/v3/users/c00d5055034d465f90a1cde955a86c86` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:32.384382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c00d5055034d465f90a1cde955a86c86)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:32.386333 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:32.432695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c00d5055034d465f90a1cde955a86c86'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:32.440837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c00d5055034d465f90a1cde955a86c86'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:32.441539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c00d5055034d465f90a1cde955a86c86'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:32.442184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c00d5055034d465f90a1cde955a86c86'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:32.449113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c00d5055034d465f90a1cde955a86c86'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:32.467097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1038/2096] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:32 2022] DELETE /identity/v3/users/c00d5055034d465f90a1cde955a86c86 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:32.471823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf540d29-fd24-411f-ab8c-e118aa70046a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:32.472290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf540d29-fd24-411f-ab8c-e118aa70046a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:32.472396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf540d29-fd24-411f-ab8c-e118aa70046a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:32.492488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bf540d29-fd24-411f-ab8c-e118aa70046a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:32.493698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bf540d29-fd24-411f-ab8c-e118aa70046a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:32.555036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1059/2097] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:32.564508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:32.565973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:32.626452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f259c-6647-41d6-a254-db63e3ef28a3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:32.631102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79c810-769d-4212-a9ff-dcd0579294ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:32.631654 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79c810-769d-4212-a9ff-dcd0579294ea service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:32.631935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79c810-769d-4212-a9ff-dcd0579294ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:32.632209 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79c810-769d-4212-a9ff-dcd0579294ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:32.643207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79c810-769d-4212-a9ff-dcd0579294ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:32.646685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e79c810-769d-4212-a9ff-dcd0579294ea service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:32.666099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1039/2098] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:33.306110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf540d29-fd24-411f-ab8c-e118aa70046a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:33.307546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bf540d29-fd24-411f-ab8c-e118aa70046a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:33.326387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf540d29-fd24-411f-ab8c-e118aa70046a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:33.331227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:33.331903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:33.332688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:33.333168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] PATH_INFO: `/v3/projects/df9f16a631fc4268b3ad70bc488ef7f5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:33.334628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=df9f16a631fc4268b3ad70bc488ef7f5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:33.336506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:33.391974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df9f16a631fc4268b3ad70bc488ef7f5', 'request_id': 'req-6420273f-7e7e-48a3-b2c0-701ca59469fb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:33.401323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df9f16a631fc4268b3ad70bc488ef7f5', 'request_id': 'req-6420273f-7e7e-48a3-b2c0-701ca59469fb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:33.407405 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df9f16a631fc4268b3ad70bc488ef7f5', 'request_id': 'req-6420273f-7e7e-48a3-b2c0-701ca59469fb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:33.419037 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] The token cache is being invalidate because project df9f16a631fc4268b3ad70bc488ef7f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:33.419324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:33.419987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:33.421509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1060/2099] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:33 2022] DELETE /identity/v3/projects/df9f16a631fc4268b3ad70bc488ef7f5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:33.426652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79c810-769d-4212-a9ff-dcd0579294ea service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:33.436322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79c810-769d-4212-a9ff-dcd0579294ea service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:33.464047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:33.510506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:33.511140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:33.511414 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:33.511688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] PATH_INFO: `/v3/users/d4960670fe3d4a88a6d8a163428c437b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:33.512872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4960670fe3d4a88a6d8a163428c437b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:33.514936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:33.572099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4960670fe3d4a88a6d8a163428c437b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:33.583257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4960670fe3d4a88a6d8a163428c437b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:33.584041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4960670fe3d4a88a6d8a163428c437b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:33.584867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4960670fe3d4a88a6d8a163428c437b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:33.594168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f4ddce1-1a51-4373-9d32-7dfe70442008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4960670fe3d4a88a6d8a163428c437b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:33.615225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1040/2100] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:33 2022] DELETE /identity/v3/users/d4960670fe3d4a88a6d8a163428c437b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:33.903743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:33.906080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:33.947412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420273f-7e7e-48a3-b2c0-701ca59469fb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:34.020010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:34.020819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.021102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.021370 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] PATH_INFO: `/v3/projects/f5c89f5623c14f9296bc340f0ea0299a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.022558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5c89f5623c14f9296bc340f0ea0299a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:34.024517 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:34.060340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:34.080923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c89f5623c14f9296bc340f0ea0299a', 'request_id': 'req-815a0779-5f7e-4225-9d3a-153553a030e8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:34.090507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c89f5623c14f9296bc340f0ea0299a', 'request_id': 'req-815a0779-5f7e-4225-9d3a-153553a030e8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:34.096478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c89f5623c14f9296bc340f0ea0299a', 'request_id': 'req-815a0779-5f7e-4225-9d3a-153553a030e8'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:34.107626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] The token cache is being invalidate because project f5c89f5623c14f9296bc340f0ea0299a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:34.107919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:34.108560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-815a0779-5f7e-4225-9d3a-153553a030e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:34.110109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1061/2101] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:33 2022] DELETE /identity/v3/projects/f5c89f5623c14f9296bc340f0ea0299a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:34.115580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a48fd-58f6-4c10-8897-5697f8ec1158 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.115868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a48fd-58f6-4c10-8897-5697f8ec1158 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.116146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a48fd-58f6-4c10-8897-5697f8ec1158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.136362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-945a48fd-58f6-4c10-8897-5697f8ec1158 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:34.137577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-945a48fd-58f6-4c10-8897-5697f8ec1158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:34.203086 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1041/2102] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:34.211428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd80c-5f20-4d8b-b6fa-3d57dab33f35 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.211716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd80c-5f20-4d8b-b6fa-3d57dab33f35 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.211995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549dd80c-5f20-4d8b-b6fa-3d57dab33f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.234234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-549dd80c-5f20-4d8b-b6fa-3d57dab33f35 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:34.235428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-549dd80c-5f20-4d8b-b6fa-3d57dab33f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:34.244928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1062/2103] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:34.249171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a48fd-58f6-4c10-8897-5697f8ec1158 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:34.260397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a48fd-58f6-4c10-8897-5697f8ec1158 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:34.265662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5aaf31-ba38-4270-bf7f-f21bc910bef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:34.266299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aaf31-ba38-4270-bf7f-f21bc910bef9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.266809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aaf31-ba38-4270-bf7f-f21bc910bef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.267545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aaf31-ba38-4270-bf7f-f21bc910bef9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.267545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5aaf31-ba38-4270-bf7f-f21bc910bef9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:10:34.267812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5aaf31-ba38-4270-bf7f-f21bc910bef9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:34.269478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5aaf31-ba38-4270-bf7f-f21bc910bef9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:34.283020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1042/2104] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:10:34.287649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dd80c-5f20-4d8b-b6fa-3d57dab33f35 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:34.300532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549dd80c-5f20-4d8b-b6fa-3d57dab33f35 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:34.306021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34693b86-aba3-4a52-89ab-f35ee2d852b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:34.306678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34693b86-aba3-4a52-89ab-f35ee2d852b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.306961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34693b86-aba3-4a52-89ab-f35ee2d852b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.307234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34693b86-aba3-4a52-89ab-f35ee2d852b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.307947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34693b86-aba3-4a52-89ab-f35ee2d852b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:34.310075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34693b86-aba3-4a52-89ab-f35ee2d852b9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:34.343393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1063/2105] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:34 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:34.347812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5aaf31-ba38-4270-bf7f-f21bc910bef9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:34.358771 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5aaf31-ba38-4270-bf7f-f21bc910bef9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:34.398117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:34.456141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb79920-5121-4722-b891-6ec74c6253a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:34.456809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb79920-5121-4722-b891-6ec74c6253a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.458372 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb79920-5121-4722-b891-6ec74c6253a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.458661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb79920-5121-4722-b891-6ec74c6253a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.459319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb79920-5121-4722-b891-6ec74c6253a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:34.461295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb79920-5121-4722-b891-6ec74c6253a4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:34.487021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1043/2106] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:34 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:10:34.491951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34693b86-aba3-4a52-89ab-f35ee2d852b9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:34.502945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34693b86-aba3-4a52-89ab-f35ee2d852b9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:34.508588 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848ca4f-2c49-4750-a319-b6afe3c02d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:34.509216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848ca4f-2c49-4750-a319-b6afe3c02d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.509492 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848ca4f-2c49-4750-a319-b6afe3c02d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.509786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848ca4f-2c49-4750-a319-b6afe3c02d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.510467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848ca4f-2c49-4750-a319-b6afe3c02d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:34.512088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848ca4f-2c49-4750-a319-b6afe3c02d89 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:34.529172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1064/2107] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:34.530605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb79920-5121-4722-b891-6ec74c6253a4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:34.542389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb79920-5121-4722-b891-6ec74c6253a4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:34.547465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24baef9-33ca-4d3e-abed-1f589c0481f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:34.548091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24baef9-33ca-4d3e-abed-1f589c0481f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.548368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24baef9-33ca-4d3e-abed-1f589c0481f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.548646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24baef9-33ca-4d3e-abed-1f589c0481f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.549298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24baef9-33ca-4d3e-abed-1f589c0481f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:34.551140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24baef9-33ca-4d3e-abed-1f589c0481f6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:34.562655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1044/2108] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:34.568087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848ca4f-2c49-4750-a319-b6afe3c02d89 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:34.579425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848ca4f-2c49-4750-a319-b6afe3c02d89 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:34.586122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e289dcb-1217-445d-9c13-4f51b28023d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:34.586868 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e289dcb-1217-445d-9c13-4f51b28023d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.587209 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e289dcb-1217-445d-9c13-4f51b28023d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.587526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e289dcb-1217-445d-9c13-4f51b28023d3 admin admin] PATH_INFO: `/v3/projects/6719d3bb31eb4add805aa308f7c4956e/users/85231d5bb195471f8766e2d4ebc98ce9/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.604663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e289dcb-1217-445d-9c13-4f51b28023d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6719d3bb31eb4add805aa308f7c4956e, user_id=85231d5bb195471f8766e2d4ebc98ce9, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:34.607399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e289dcb-1217-445d-9c13-4f51b28023d3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:34.622841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1065/2109] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:34 2022] PUT /identity/v3/projects/6719d3bb31eb4add805aa308f7c4956e/users/85231d5bb195471f8766e2d4ebc98ce9/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:34.637723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97848d4f-bb08-4e28-9607-b4f490d2801b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.638273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97848d4f-bb08-4e28-9607-b4f490d2801b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.638561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97848d4f-bb08-4e28-9607-b4f490d2801b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.684848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-97848d4f-bb08-4e28-9607-b4f490d2801b None None] MFA Rules not processed for user `85231d5bb195471f8766e2d4ebc98ce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:34.687184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-97848d4f-bb08-4e28-9607-b4f490d2801b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:34.780411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1045/2110] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:34.785353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e289dcb-1217-445d-9c13-4f51b28023d3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:34.855209 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e289dcb-1217-445d-9c13-4f51b28023d3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:34.860886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecac904-e989-4450-b4f2-8d42acc37e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:34.861499 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecac904-e989-4450-b4f2-8d42acc37e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.861808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecac904-e989-4450-b4f2-8d42acc37e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.862079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecac904-e989-4450-b4f2-8d42acc37e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.862712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecac904-e989-4450-b4f2-8d42acc37e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:34.865409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecac904-e989-4450-b4f2-8d42acc37e3d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:34.898096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1066/2111] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:34 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:34.902694 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97848d4f-bb08-4e28-9607-b4f490d2801b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:34.915381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97848d4f-bb08-4e28-9607-b4f490d2801b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:34.984442 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207caac2-8963-4c61-b73e-fd1a5c8a0681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:34.985181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207caac2-8963-4c61-b73e-fd1a5c8a0681 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:34.985533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207caac2-8963-4c61-b73e-fd1a5c8a0681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:34.985909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207caac2-8963-4c61-b73e-fd1a5c8a0681 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:34.986621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207caac2-8963-4c61-b73e-fd1a5c8a0681 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:34.989358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207caac2-8963-4c61-b73e-fd1a5c8a0681 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:35.028229 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1046/2112] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:34 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:35.029134 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecac904-e989-4450-b4f2-8d42acc37e3d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:35.045676 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecac904-e989-4450-b4f2-8d42acc37e3d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:35.051864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a6884-1744-4ca6-899d-b5ac9975c36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:35.052683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a6884-1744-4ca6-899d-b5ac9975c36b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.053648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a6884-1744-4ca6-899d-b5ac9975c36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.054273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a6884-1744-4ca6-899d-b5ac9975c36b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.055661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a6884-1744-4ca6-899d-b5ac9975c36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:35.064622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a6884-1744-4ca6-899d-b5ac9975c36b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:35.081104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1067/2113] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:35.084997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207caac2-8963-4c61-b73e-fd1a5c8a0681 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:35.102317 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207caac2-8963-4c61-b73e-fd1a5c8a0681 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:35.115746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12178fe-a5d6-4286-b601-e4bf02220b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:35.116383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12178fe-a5d6-4286-b601-e4bf02220b91 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.117957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12178fe-a5d6-4286-b601-e4bf02220b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.117957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12178fe-a5d6-4286-b601-e4bf02220b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.119342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12178fe-a5d6-4286-b601-e4bf02220b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:35.122132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12178fe-a5d6-4286-b601-e4bf02220b91 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:35.152223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1047/2114] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:35.153550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a6884-1744-4ca6-899d-b5ac9975c36b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:35.166017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a6884-1744-4ca6-899d-b5ac9975c36b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:35.171340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d75774-9ddd-4cef-a6c5-5a77d8c191a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:35.172122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d75774-9ddd-4cef-a6c5-5a77d8c191a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.172613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d75774-9ddd-4cef-a6c5-5a77d8c191a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.173131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d75774-9ddd-4cef-a6c5-5a77d8c191a6 admin admin] PATH_INFO: `/v3/projects/7d9ab4038dff4081959298e94843edf3/users/9638969c60224872bf73d2db7bc5d183/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.195999 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d75774-9ddd-4cef-a6c5-5a77d8c191a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d9ab4038dff4081959298e94843edf3, user_id=9638969c60224872bf73d2db7bc5d183, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:35.199859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d75774-9ddd-4cef-a6c5-5a77d8c191a6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:35.209646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:35.216692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1068/2115] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:35 2022] PUT /identity/v3/projects/7d9ab4038dff4081959298e94843edf3/users/9638969c60224872bf73d2db7bc5d183/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:35.228748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1717e4-6740-4fd7-bc92-4b70d7cadc79 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.229065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1717e4-6740-4fd7-bc92-4b70d7cadc79 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.229342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1717e4-6740-4fd7-bc92-4b70d7cadc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.270137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3b1717e4-6740-4fd7-bc92-4b70d7cadc79 None None] MFA Rules not processed for user `9638969c60224872bf73d2db7bc5d183`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:35.271793 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3b1717e4-6740-4fd7-bc92-4b70d7cadc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:35.276100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:35.345997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1048/2116] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:35.351180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d75774-9ddd-4cef-a6c5-5a77d8c191a6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:35.439719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d75774-9ddd-4cef-a6c5-5a77d8c191a6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:35.445058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b95dda-7654-4b9e-8a56-b9ef2f4e074f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:35.445714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b95dda-7654-4b9e-8a56-b9ef2f4e074f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.446034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b95dda-7654-4b9e-8a56-b9ef2f4e074f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.446371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b95dda-7654-4b9e-8a56-b9ef2f4e074f admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.447299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b95dda-7654-4b9e-8a56-b9ef2f4e074f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:35.449255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b95dda-7654-4b9e-8a56-b9ef2f4e074f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:35.491000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1069/2117] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:35 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:10:35.492453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1717e4-6740-4fd7-bc92-4b70d7cadc79 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:35.509384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1717e4-6740-4fd7-bc92-4b70d7cadc79 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:35.580696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f1ec8-4d14-4d01-ac7f-ed3f7a0abadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:35.581315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f1ec8-4d14-4d01-ac7f-ed3f7a0abadf admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.581595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f1ec8-4d14-4d01-ac7f-ed3f7a0abadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.583027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f1ec8-4d14-4d01-ac7f-ed3f7a0abadf admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.583376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f1ec8-4d14-4d01-ac7f-ed3f7a0abadf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:35.585617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f1ec8-4d14-4d01-ac7f-ed3f7a0abadf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:35.610294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1049/2118] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:35 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:35.615221 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b95dda-7654-4b9e-8a56-b9ef2f4e074f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:35.627087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b95dda-7654-4b9e-8a56-b9ef2f4e074f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:35.632779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfe8d1-b149-49e4-80ec-d98d44f22f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:35.633424 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe8d1-b149-49e4-80ec-d98d44f22f5d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.633704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe8d1-b149-49e4-80ec-d98d44f22f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.634002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcfe8d1-b149-49e4-80ec-d98d44f22f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.634659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfe8d1-b149-49e4-80ec-d98d44f22f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:35.636279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcfe8d1-b149-49e4-80ec-d98d44f22f5d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:35.648299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1070/2119] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:35.652712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f1ec8-4d14-4d01-ac7f-ed3f7a0abadf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:35.664569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f1ec8-4d14-4d01-ac7f-ed3f7a0abadf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:35.669554 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c2fd7-abf7-4baa-b9e7-710dfc8e5872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:35.670337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c2fd7-abf7-4baa-b9e7-710dfc8e5872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.670681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c2fd7-abf7-4baa-b9e7-710dfc8e5872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.671041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c2fd7-abf7-4baa-b9e7-710dfc8e5872 admin admin] PATH_INFO: `/v3/projects/3906b614ecd64109b51f0a634b7a35c5/users/02b1c82f0caf42f794759894ad01b850/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.689514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c2fd7-abf7-4baa-b9e7-710dfc8e5872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3906b614ecd64109b51f0a634b7a35c5, user_id=02b1c82f0caf42f794759894ad01b850, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:35.692525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c2fd7-abf7-4baa-b9e7-710dfc8e5872 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:35.712849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1050/2120] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:35 2022] PUT /identity/v3/projects/3906b614ecd64109b51f0a634b7a35c5/users/02b1c82f0caf42f794759894ad01b850/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:35.718776 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfe8d1-b149-49e4-80ec-d98d44f22f5d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:35.731102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcfe8d1-b149-49e4-80ec-d98d44f22f5d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:35.805466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566800a3-a8fd-4555-b98d-8cf7c7b1bfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:35.806137 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566800a3-a8fd-4555-b98d-8cf7c7b1bfea admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.806419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566800a3-a8fd-4555-b98d-8cf7c7b1bfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.806698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566800a3-a8fd-4555-b98d-8cf7c7b1bfea admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.807353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566800a3-a8fd-4555-b98d-8cf7c7b1bfea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:35.808963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566800a3-a8fd-4555-b98d-8cf7c7b1bfea admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:35.819969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1071/2121] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:35.824373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c2fd7-abf7-4baa-b9e7-710dfc8e5872 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:35.835873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c2fd7-abf7-4baa-b9e7-710dfc8e5872 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:35.840655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d802d-4240-425b-b8f5-2872e3155f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:35.841314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d802d-4240-425b-b8f5-2872e3155f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.841588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d802d-4240-425b-b8f5-2872e3155f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.841898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d802d-4240-425b-b8f5-2872e3155f1f admin admin] PATH_INFO: `/v3/projects/3906b614ecd64109b51f0a634b7a35c5/users/02b1c82f0caf42f794759894ad01b850/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.843824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d802d-4240-425b-b8f5-2872e3155f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3906b614ecd64109b51f0a634b7a35c5, user_id=02b1c82f0caf42f794759894ad01b850, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:35.846497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d802d-4240-425b-b8f5-2872e3155f1f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:35.858371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1051/2122] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:35 2022] PUT /identity/v3/projects/3906b614ecd64109b51f0a634b7a35c5/users/02b1c82f0caf42f794759894ad01b850/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:35.870382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e6f48-d6ab-4421-925e-b4bca84b5985 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:35.870580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e6f48-d6ab-4421-925e-b4bca84b5985 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:35.870758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e6f48-d6ab-4421-925e-b4bca84b5985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:35.906647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5b6e6f48-d6ab-4421-925e-b4bca84b5985 None None] MFA Rules not processed for user `02b1c82f0caf42f794759894ad01b850`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:35.908128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5b6e6f48-d6ab-4421-925e-b4bca84b5985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:35.978129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:35.989531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1072/2123] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:36.001859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d802d-4240-425b-b8f5-2872e3155f1f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:36.004688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa1d802d-4240-425b-b8f5-2872e3155f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:36.072992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d802d-4240-425b-b8f5-2872e3155f1f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:36.076626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96439008-742e-4785-a95d-cf6895bec5cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:36.077418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439008-742e-4785-a95d-cf6895bec5cb service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:36.077704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439008-742e-4785-a95d-cf6895bec5cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:36.078015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439008-742e-4785-a95d-cf6895bec5cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:36.088327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96439008-742e-4785-a95d-cf6895bec5cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:36.090058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96439008-742e-4785-a95d-cf6895bec5cb service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:36.095481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:36.102299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1052/2124] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:10:36.147142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e6f48-d6ab-4421-925e-b4bca84b5985 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:36.157096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e6f48-d6ab-4421-925e-b4bca84b5985 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:36.161963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573dd9bf-9d91-48fb-9724-e680d6d91aba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:36.162555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dd9bf-9d91-48fb-9724-e680d6d91aba service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:36.162843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dd9bf-9d91-48fb-9724-e680d6d91aba service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:36.163095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573dd9bf-9d91-48fb-9724-e680d6d91aba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:36.226779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573dd9bf-9d91-48fb-9724-e680d6d91aba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:36.228372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573dd9bf-9d91-48fb-9724-e680d6d91aba service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:36.238500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1073/2125] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:37.344598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96439008-742e-4785-a95d-cf6895bec5cb service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:37.346092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-96439008-742e-4785-a95d-cf6895bec5cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:37.414234 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96439008-742e-4785-a95d-cf6895bec5cb service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:37.418954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:37.419578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:37.420092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:37.420369 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] PATH_INFO: `/v3/users/b0091db241b24ee3a6e56abd5cccca5a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:37.421501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0091db241b24ee3a6e56abd5cccca5a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:37.423466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:37.478083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0091db241b24ee3a6e56abd5cccca5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:37.486002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0091db241b24ee3a6e56abd5cccca5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:37.486618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0091db241b24ee3a6e56abd5cccca5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:37.487247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0091db241b24ee3a6e56abd5cccca5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:37.495531 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0091db241b24ee3a6e56abd5cccca5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:37.513688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1053/2126] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:37 2022] DELETE /identity/v3/users/b0091db241b24ee3a6e56abd5cccca5a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:37.518165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b933d0-4935-41a6-afb9-1ddedd187894 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:37.518676 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b933d0-4935-41a6-afb9-1ddedd187894 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:37.518758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b933d0-4935-41a6-afb9-1ddedd187894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:37.538652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e9b933d0-4935-41a6-afb9-1ddedd187894 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:37.539865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e9b933d0-4935-41a6-afb9-1ddedd187894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:37.604321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1074/2127] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:37.615026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:37.616547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:37.683319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02da0a4-5d5e-4eba-8972-e6e1158893a6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:37.687168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a3a19-a6f7-4e27-983c-f6da82ad3abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:37.687785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a3a19-a6f7-4e27-983c-f6da82ad3abf service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:37.688071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a3a19-a6f7-4e27-983c-f6da82ad3abf service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:37.688351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a3a19-a6f7-4e27-983c-f6da82ad3abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:37.699028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a3a19-a6f7-4e27-983c-f6da82ad3abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:37.700488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a3a19-a6f7-4e27-983c-f6da82ad3abf service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:37.711906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1054/2128] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:10:38.005100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b933d0-4935-41a6-afb9-1ddedd187894 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:38.007298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e9b933d0-4935-41a6-afb9-1ddedd187894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:38.020768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b933d0-4935-41a6-afb9-1ddedd187894 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:38.032722 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:38.033395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:38.033674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:38.033981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] PATH_INFO: `/v3/projects/2c3c0cab80f34f37b510ce8b258edb6b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:38.035996 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c3c0cab80f34f37b510ce8b258edb6b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:38.038117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:38.092096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c3c0cab80f34f37b510ce8b258edb6b', 'request_id': 'req-5bec6d4b-76f8-44ba-bcac-617626838a99'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.102295 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c3c0cab80f34f37b510ce8b258edb6b', 'request_id': 'req-5bec6d4b-76f8-44ba-bcac-617626838a99'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.109945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c3c0cab80f34f37b510ce8b258edb6b', 'request_id': 'req-5bec6d4b-76f8-44ba-bcac-617626838a99'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.123321 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] The token cache is being invalidate because project 2c3c0cab80f34f37b510ce8b258edb6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:38.123606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.124278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.125819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1075/2129] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:38 2022] DELETE /identity/v3/projects/2c3c0cab80f34f37b510ce8b258edb6b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:38.130565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a3a19-a6f7-4e27-983c-f6da82ad3abf service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:38.140932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a3a19-a6f7-4e27-983c-f6da82ad3abf service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:38.162461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:38.211937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:38.212628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:38.212904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:38.213178 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] PATH_INFO: `/v3/users/623134c872514fce9c76e2e995cf4c2e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:38.214565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] RBAC: Authorizing `identity:delete_user(user_id=623134c872514fce9c76e2e995cf4c2e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:38.216530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:38.287951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '623134c872514fce9c76e2e995cf4c2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.296633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '623134c872514fce9c76e2e995cf4c2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.297438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '623134c872514fce9c76e2e995cf4c2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.298105 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '623134c872514fce9c76e2e995cf4c2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.308697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-52f84761-00b0-4e09-94fd-680b6ce361de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '623134c872514fce9c76e2e995cf4c2e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.332377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1055/2130] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:38 2022] DELETE /identity/v3/users/623134c872514fce9c76e2e995cf4c2e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:38.665099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:38.666646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:38.680972 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec6d4b-76f8-44ba-bcac-617626838a99 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:38.696479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:38.749320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:38.750072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:38.750422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:38.750649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] PATH_INFO: `/v3/projects/e8bfc56a54004846a518d6058e2fbaab` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:38.751803 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8bfc56a54004846a518d6058e2fbaab)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:38.754845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:38.831744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8bfc56a54004846a518d6058e2fbaab', 'request_id': 'req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.841615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8bfc56a54004846a518d6058e2fbaab', 'request_id': 'req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.849313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8bfc56a54004846a518d6058e2fbaab', 'request_id': 'req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.870325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] The token cache is being invalidate because project e8bfc56a54004846a518d6058e2fbaab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:38.870858 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.871953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85b2b75c-5da3-416d-a4fe-1a0e025ffb49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:38.874605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1076/2131] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:38 2022] DELETE /identity/v3/projects/e8bfc56a54004846a518d6058e2fbaab => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:38.885729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147b272-d034-47c3-b353-f482e95355f0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:38.886295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147b272-d034-47c3-b353-f482e95355f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:38.886854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147b272-d034-47c3-b353-f482e95355f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:38.918652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1147b272-d034-47c3-b353-f482e95355f0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:38.919918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1147b272-d034-47c3-b353-f482e95355f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:38.991015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1056/2132] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:38.999246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab081399-5327-4ff6-ae1e-21059159b094 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:38.999537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab081399-5327-4ff6-ae1e-21059159b094 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:38.999816 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab081399-5327-4ff6-ae1e-21059159b094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:39.020811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ab081399-5327-4ff6-ae1e-21059159b094 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:39.022062 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ab081399-5327-4ff6-ae1e-21059159b094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:39.030328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1077/2133] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:39.034579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147b272-d034-47c3-b353-f482e95355f0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:39.044618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147b272-d034-47c3-b353-f482e95355f0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:39.049404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d47234-82e8-4eeb-a74d-aed0c7488d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:39.050065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d47234-82e8-4eeb-a74d-aed0c7488d4f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:39.050347 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d47234-82e8-4eeb-a74d-aed0c7488d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:39.051160 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d47234-82e8-4eeb-a74d-aed0c7488d4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:39.051867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d47234-82e8-4eeb-a74d-aed0c7488d4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:10:39.052235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d47234-82e8-4eeb-a74d-aed0c7488d4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:39.053864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d47234-82e8-4eeb-a74d-aed0c7488d4f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:39.065620 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1057/2134] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:10:39.070978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab081399-5327-4ff6-ae1e-21059159b094 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:39.080195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab081399-5327-4ff6-ae1e-21059159b094 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:39.085475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085f1e9-81f8-4c33-9b5c-59c3cb4fca7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:39.086108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f1e9-81f8-4c33-9b5c-59c3cb4fca7f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:39.086387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f1e9-81f8-4c33-9b5c-59c3cb4fca7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:39.086660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085f1e9-81f8-4c33-9b5c-59c3cb4fca7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:39.087310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085f1e9-81f8-4c33-9b5c-59c3cb4fca7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:39.089540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085f1e9-81f8-4c33-9b5c-59c3cb4fca7f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:39.118445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1078/2135] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:39 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:39.122798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d47234-82e8-4eeb-a74d-aed0c7488d4f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:39.133266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d47234-82e8-4eeb-a74d-aed0c7488d4f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:39.159095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:39.193166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17894565-df18-4f32-8e03-a8bf93fbbe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:39.193678 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17894565-df18-4f32-8e03-a8bf93fbbe40 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:39.193987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17894565-df18-4f32-8e03-a8bf93fbbe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:39.194264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17894565-df18-4f32-8e03-a8bf93fbbe40 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:39.194890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17894565-df18-4f32-8e03-a8bf93fbbe40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:39.196828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17894565-df18-4f32-8e03-a8bf93fbbe40 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:39.218491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1058/2136] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:39 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:39.223886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085f1e9-81f8-4c33-9b5c-59c3cb4fca7f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:39.234540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085f1e9-81f8-4c33-9b5c-59c3cb4fca7f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:39.239522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445760e-99e2-43ec-882f-3c21f536137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:39.240133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445760e-99e2-43ec-882f-3c21f536137f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:39.240414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445760e-99e2-43ec-882f-3c21f536137f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:39.240688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4445760e-99e2-43ec-882f-3c21f536137f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:39.241336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445760e-99e2-43ec-882f-3c21f536137f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:39.242967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4445760e-99e2-43ec-882f-3c21f536137f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:39.253433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1079/2137] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:39.257655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17894565-df18-4f32-8e03-a8bf93fbbe40 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:39.267839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17894565-df18-4f32-8e03-a8bf93fbbe40 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:39.273226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693f9d2-23fa-47d5-8fab-3c05e2a2910a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:39.273873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693f9d2-23fa-47d5-8fab-3c05e2a2910a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:39.274154 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693f9d2-23fa-47d5-8fab-3c05e2a2910a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:39.274429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693f9d2-23fa-47d5-8fab-3c05e2a2910a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:39.275088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693f9d2-23fa-47d5-8fab-3c05e2a2910a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:39.276785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693f9d2-23fa-47d5-8fab-3c05e2a2910a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:39.288484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1059/2138] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:39.293346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445760e-99e2-43ec-882f-3c21f536137f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:39.304328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4445760e-99e2-43ec-882f-3c21f536137f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:39.309129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048197ac-9975-47e3-96f6-79f27ecad219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:39.309828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048197ac-9975-47e3-96f6-79f27ecad219 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:39.310112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048197ac-9975-47e3-96f6-79f27ecad219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:39.310418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048197ac-9975-47e3-96f6-79f27ecad219 admin admin] PATH_INFO: `/v3/projects/fcd0c070a1eb4d96905efb70d570bcb2/users/24f86342b11b499d84ffa217312e2184/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:39.326251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048197ac-9975-47e3-96f6-79f27ecad219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcd0c070a1eb4d96905efb70d570bcb2, user_id=24f86342b11b499d84ffa217312e2184, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:39.329013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048197ac-9975-47e3-96f6-79f27ecad219 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:39.344416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1080/2139] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:39 2022] PUT /identity/v3/projects/fcd0c070a1eb4d96905efb70d570bcb2/users/24f86342b11b499d84ffa217312e2184/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:39.357035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ee8e-c03f-4cc9-aeca-a67838d74ccb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:39.357382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ee8e-c03f-4cc9-aeca-a67838d74ccb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:39.357736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa19ee8e-c03f-4cc9-aeca-a67838d74ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:39.389478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-aa19ee8e-c03f-4cc9-aeca-a67838d74ccb None None] MFA Rules not processed for user `24f86342b11b499d84ffa217312e2184`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:39.390872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aa19ee8e-c03f-4cc9-aeca-a67838d74ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:39.470400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1060/2140] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:39.483733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048197ac-9975-47e3-96f6-79f27ecad219 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:39.485626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-048197ac-9975-47e3-96f6-79f27ecad219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:39.540515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:39.554437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048197ac-9975-47e3-96f6-79f27ecad219 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:39.558775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51316369-558b-4efa-a1e8-c13856d7a292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:39.559439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51316369-558b-4efa-a1e8-c13856d7a292 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:39.559723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51316369-558b-4efa-a1e8-c13856d7a292 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:39.560081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51316369-558b-4efa-a1e8-c13856d7a292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:39.570673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51316369-558b-4efa-a1e8-c13856d7a292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:39.572158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51316369-558b-4efa-a1e8-c13856d7a292 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:39.587434 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1081/2141] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:49.466913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:49.578469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:50.346493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19ee8e-c03f-4cc9-aeca-a67838d74ccb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:50.348813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aa19ee8e-c03f-4cc9-aeca-a67838d74ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:50.427773 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa19ee8e-c03f-4cc9-aeca-a67838d74ccb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:50.432981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:50.433672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:50.433988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:50.434273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] PATH_INFO: `/v3/users/85231d5bb195471f8766e2d4ebc98ce9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:50.435549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=85231d5bb195471f8766e2d4ebc98ce9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:50.437523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:50.471119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51316369-558b-4efa-a1e8-c13856d7a292 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:50.490549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51316369-558b-4efa-a1e8-c13856d7a292 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:50.500165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:50.501249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:50.502001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:50.502667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] PATH_INFO: `/v3/users/24f86342b11b499d84ffa217312e2184` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:50.505053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] RBAC: Authorizing `identity:delete_user(user_id=24f86342b11b499d84ffa217312e2184)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:50.507851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:50.517520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85231d5bb195471f8766e2d4ebc98ce9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:50.531228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85231d5bb195471f8766e2d4ebc98ce9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:50.532350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85231d5bb195471f8766e2d4ebc98ce9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:50.533411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85231d5bb195471f8766e2d4ebc98ce9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:50.551846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8b4ab622-46bd-4b8b-9d9d-7c4640dc0f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85231d5bb195471f8766e2d4ebc98ce9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:50.572268 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24f86342b11b499d84ffa217312e2184'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:50.573523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24f86342b11b499d84ffa217312e2184'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:50.574494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24f86342b11b499d84ffa217312e2184'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:50.577131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1061/2142] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:50 2022] DELETE /identity/v3/users/85231d5bb195471f8766e2d4ebc98ce9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:50.583384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b353600-930f-4eba-a51b-0c31241ddb89 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:50.583860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b353600-930f-4eba-a51b-0c31241ddb89 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:50.584335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b353600-930f-4eba-a51b-0c31241ddb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:50.586587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24f86342b11b499d84ffa217312e2184'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:50.607428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-09a81e49-75e8-41c3-9e36-764e1f0a877d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24f86342b11b499d84ffa217312e2184'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:50.621866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3b353600-930f-4eba-a51b-0c31241ddb89 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:50.624305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3b353600-930f-4eba-a51b-0c31241ddb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:50.631794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1082/2143] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:50 2022] DELETE /identity/v3/users/24f86342b11b499d84ffa217312e2184 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:50.638516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989535ab-6a58-41ff-834c-c282ac1a5356 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:50.638975 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989535ab-6a58-41ff-834c-c282ac1a5356 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:50.639437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989535ab-6a58-41ff-834c-c282ac1a5356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:50.670630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-989535ab-6a58-41ff-834c-c282ac1a5356 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:50.672438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-989535ab-6a58-41ff-834c-c282ac1a5356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:50.741082 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1062/2144] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:50.758223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b353600-930f-4eba-a51b-0c31241ddb89 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:50.760679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3b353600-930f-4eba-a51b-0c31241ddb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:50.793826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1083/2145] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:50.809339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989535ab-6a58-41ff-834c-c282ac1a5356 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:50.812283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-989535ab-6a58-41ff-834c-c282ac1a5356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:50.827808 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:50.933628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b353600-930f-4eba-a51b-0c31241ddb89 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:50.983129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989535ab-6a58-41ff-834c-c282ac1a5356 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:50.989286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54521-7f1e-4638-87e4-3717a951b4f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:50.990104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54521-7f1e-4638-87e4-3717a951b4f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:50.990511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54521-7f1e-4638-87e4-3717a951b4f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:50.990866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54521-7f1e-4638-87e4-3717a951b4f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:51.005206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54521-7f1e-4638-87e4-3717a951b4f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:51.010595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54521-7f1e-4638-87e4-3717a951b4f1 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:51.041232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be324a3e-77b3-44b7-9992-20a85fddece1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:51.043169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be324a3e-77b3-44b7-9992-20a85fddece1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:51.043799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1084/2146] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:51.046189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be324a3e-77b3-44b7-9992-20a85fddece1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:51.046581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be324a3e-77b3-44b7-9992-20a85fddece1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:51.067942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be324a3e-77b3-44b7-9992-20a85fddece1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:51.072481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be324a3e-77b3-44b7-9992-20a85fddece1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:51.094241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1063/2147] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:51.832668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54521-7f1e-4638-87e4-3717a951b4f1 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:51.834544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-22d54521-7f1e-4638-87e4-3717a951b4f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:51.849034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54521-7f1e-4638-87e4-3717a951b4f1 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:51.855079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:51.855916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:51.856304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:51.856721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] PATH_INFO: `/v3/projects/6719d3bb31eb4add805aa308f7c4956e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:51.858050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6719d3bb31eb4add805aa308f7c4956e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:51.860299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:51.881574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:51.922613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be324a3e-77b3-44b7-9992-20a85fddece1 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:51.924180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-be324a3e-77b3-44b7-9992-20a85fddece1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:51.939673 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be324a3e-77b3-44b7-9992-20a85fddece1 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:51.955001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6719d3bb31eb4add805aa308f7c4956e', 'request_id': 'req-544f1a59-32c4-44c4-b63f-4d061988d0dc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:51.967231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6719d3bb31eb4add805aa308f7c4956e', 'request_id': 'req-544f1a59-32c4-44c4-b63f-4d061988d0dc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:51.977872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6719d3bb31eb4add805aa308f7c4956e', 'request_id': 'req-544f1a59-32c4-44c4-b63f-4d061988d0dc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:51.999188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] The token cache is being invalidate because project 6719d3bb31eb4add805aa308f7c4956e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:51.999586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.000453 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.003669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1085/2148] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:51 2022] DELETE /identity/v3/projects/6719d3bb31eb4add805aa308f7c4956e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:52.010857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:52.015152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:52.039620 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:52.040911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:52.041265 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:52.041619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] PATH_INFO: `/v3/projects/fcd0c070a1eb4d96905efb70d570bcb2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:52.043802 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcd0c070a1eb4d96905efb70d570bcb2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:52.046026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:52.172487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcd0c070a1eb4d96905efb70d570bcb2', 'request_id': 'req-b1c6dbd3-6336-47bc-90ce-b53acec37d21'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.200063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcd0c070a1eb4d96905efb70d570bcb2', 'request_id': 'req-b1c6dbd3-6336-47bc-90ce-b53acec37d21'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.226412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f1a59-32c4-44c4-b63f-4d061988d0dc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:52.246478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcd0c070a1eb4d96905efb70d570bcb2', 'request_id': 'req-b1c6dbd3-6336-47bc-90ce-b53acec37d21'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.274952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] The token cache is being invalidate because project fcd0c070a1eb4d96905efb70d570bcb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:52.276949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.279132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b1c6dbd3-6336-47bc-90ce-b53acec37d21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.282302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1064/2149] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:51 2022] DELETE /identity/v3/projects/fcd0c070a1eb4d96905efb70d570bcb2 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:52.290054 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91954fa7-27f9-4f50-b33d-7f6527b481ce None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:52.290981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91954fa7-27f9-4f50-b33d-7f6527b481ce None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:52.291575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91954fa7-27f9-4f50-b33d-7f6527b481ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:52.328994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:52.329885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:52.330279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:52.330590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] PATH_INFO: `/v3/users/9638969c60224872bf73d2db7bc5d183` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:52.333220 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9638969c60224872bf73d2db7bc5d183)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:52.338001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:52.338139 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-91954fa7-27f9-4f50-b33d-7f6527b481ce None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:52.341171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-91954fa7-27f9-4f50-b33d-7f6527b481ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:52.357863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1065/2150] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:52.369288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e741a3-96e1-451b-9644-565164fd4ad0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:52.369508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e741a3-96e1-451b-9644-565164fd4ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:52.371056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e741a3-96e1-451b-9644-565164fd4ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:52.385671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:52.415684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f7e741a3-96e1-451b-9644-565164fd4ad0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:52.417691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f7e741a3-96e1-451b-9644-565164fd4ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:52.441849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1066/2151] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:52.446930 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e741a3-96e1-451b-9644-565164fd4ad0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:52.464623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e741a3-96e1-451b-9644-565164fd4ad0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:52.475296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed4846-5ea7-4238-96f4-905473a2021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:52.476582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed4846-5ea7-4238-96f4-905473a2021d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:52.476870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed4846-5ea7-4238-96f4-905473a2021d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:52.477537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed4846-5ea7-4238-96f4-905473a2021d admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:52.478592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed4846-5ea7-4238-96f4-905473a2021d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:10:52.479043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed4846-5ea7-4238-96f4-905473a2021d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:52.481626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed4846-5ea7-4238-96f4-905473a2021d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:52.491073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9638969c60224872bf73d2db7bc5d183'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.491963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9638969c60224872bf73d2db7bc5d183'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.493407 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9638969c60224872bf73d2db7bc5d183'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.496402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:52.509836 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1067/2152] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:10:52.515661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed4846-5ea7-4238-96f4-905473a2021d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:52.515918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9638969c60224872bf73d2db7bc5d183'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.518604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-36ed4846-5ea7-4238-96f4-905473a2021d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:52.545001 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed4846-5ea7-4238-96f4-905473a2021d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:52.556438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78e5c5-f4bc-461c-a47b-c8ea6b0d6680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:52.556751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9638969c60224872bf73d2db7bc5d183'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:52.560916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e5c5-f4bc-461c-a47b-c8ea6b0d6680 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:52.562571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e5c5-f4bc-461c-a47b-c8ea6b0d6680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:52.563465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78e5c5-f4bc-461c-a47b-c8ea6b0d6680 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:52.565679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78e5c5-f4bc-461c-a47b-c8ea6b0d6680 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:52.570390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78e5c5-f4bc-461c-a47b-c8ea6b0d6680 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:52.581111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1086/2153] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:52 2022] DELETE /identity/v3/users/9638969c60224872bf73d2db7bc5d183 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:52.655049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1068/2154] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:52 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:52.655488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:52.770201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328ddf4-6260-41de-983a-f890923862f9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:52.896814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8c73b-f134-4830-92a2-f1f94c208058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:52.898000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8c73b-f134-4830-92a2-f1f94c208058 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:52.898801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8c73b-f134-4830-92a2-f1f94c208058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:52.899384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8c73b-f134-4830-92a2-f1f94c208058 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:52.900821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8c73b-f134-4830-92a2-f1f94c208058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:52.905488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8c73b-f134-4830-92a2-f1f94c208058 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:52.953561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78e5c5-f4bc-461c-a47b-c8ea6b0d6680 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:52.962041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1087/2155] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:52 2022] POST /identity/v3/users => generated 288 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:52.971976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78e5c5-f4bc-461c-a47b-c8ea6b0d6680 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:52.979952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9ba0b-336e-4e31-8b47-4aa5014f6e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:52.980933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9ba0b-336e-4e31-8b47-4aa5014f6e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:52.981194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9ba0b-336e-4e31-8b47-4aa5014f6e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:52.981643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9ba0b-336e-4e31-8b47-4aa5014f6e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:52.982658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9ba0b-336e-4e31-8b47-4aa5014f6e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:52.984485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9ba0b-336e-4e31-8b47-4aa5014f6e5e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:52.997070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1069/2156] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:53.001298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8c73b-f134-4830-92a2-f1f94c208058 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:53.002402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9ba0b-336e-4e31-8b47-4aa5014f6e5e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:53.017519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9ba0b-336e-4e31-8b47-4aa5014f6e5e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:53.018564 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a1e8c73b-f134-4830-92a2-f1f94c208058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:53.024195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6cb49f-c695-419a-8a67-98ce7c836ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:53.024913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cb49f-c695-419a-8a67-98ce7c836ea8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:53.025264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cb49f-c695-419a-8a67-98ce7c836ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:53.025614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6cb49f-c695-419a-8a67-98ce7c836ea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:53.026416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cb49f-c695-419a-8a67-98ce7c836ea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:53.028262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6cb49f-c695-419a-8a67-98ce7c836ea8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:53.048393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8c73b-f134-4830-92a2-f1f94c208058 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:53.055302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1070/2157] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:53.055302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6a738-e315-4e7a-9843-d910556ed7c0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:53.055302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6a738-e315-4e7a-9843-d910556ed7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:53.055302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a6a738-e315-4e7a-9843-d910556ed7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:53.055887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:53.055887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:53.055887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:53.055887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] PATH_INFO: `/v3/projects/7d9ab4038dff4081959298e94843edf3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:53.056432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d9ab4038dff4081959298e94843edf3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:53.058122 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:53.084496 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e6a6a738-e315-4e7a-9843-d910556ed7c0 None None] MFA Rules not processed for user `e8e345dbc2114b718c98c70844352c63`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:53.085941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e6a6a738-e315-4e7a-9843-d910556ed7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:53.133008 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d9ab4038dff4081959298e94843edf3', 'request_id': 'req-7ff26a90-12f6-419c-8873-41de7374856a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.150251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d9ab4038dff4081959298e94843edf3', 'request_id': 'req-7ff26a90-12f6-419c-8873-41de7374856a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.152681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=5,keystone:DELETE=8,keystone:SELECT=87 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:53.161722 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d9ab4038dff4081959298e94843edf3', 'request_id': 'req-7ff26a90-12f6-419c-8873-41de7374856a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.164802 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1071/2158] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:53.165659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6a738-e315-4e7a-9843-d910556ed7c0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:53.179236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a6a738-e315-4e7a-9843-d910556ed7c0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:53.184047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] The token cache is being invalidate because project 7d9ab4038dff4081959298e94843edf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:53.184441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.186220 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.187078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1088/2159] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:52 2022] DELETE /identity/v3/projects/7d9ab4038dff4081959298e94843edf3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:53.192686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:53.194747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:53.289211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff26a90-12f6-419c-8873-41de7374856a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:53.296874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:53.298637 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:53.298849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:53.299958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] PATH_INFO: `/v3/users/02b1c82f0caf42f794759894ad01b850` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:53.302669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b1c82f0caf42f794759894ad01b850)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:53.307697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:53.307913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5e4ae-f16b-4eb4-b962-4597e27260d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:53.308919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5e4ae-f16b-4eb4-b962-4597e27260d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:53.309466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5e4ae-f16b-4eb4-b962-4597e27260d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:53.310298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5e4ae-f16b-4eb4-b962-4597e27260d7 admin admin] PATH_INFO: `/v3/projects/60c73dd40d27429986f2c3cbd788f9bd/users/08e4344332a743b4aca2600b1ad4c817/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:53.330809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5e4ae-f16b-4eb4-b962-4597e27260d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60c73dd40d27429986f2c3cbd788f9bd, user_id=08e4344332a743b4aca2600b1ad4c817, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:53.333577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5e4ae-f16b-4eb4-b962-4597e27260d7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:53.366016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1072/2160] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:53 2022] PUT /identity/v3/projects/60c73dd40d27429986f2c3cbd788f9bd/users/08e4344332a743b4aca2600b1ad4c817/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:53.379866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f079134-9edc-4c40-b28b-c236af3d69be None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:53.382611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f079134-9edc-4c40-b28b-c236af3d69be None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:53.386524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f079134-9edc-4c40-b28b-c236af3d69be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:53.409376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b1c82f0caf42f794759894ad01b850'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.412103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b1c82f0caf42f794759894ad01b850'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.427435 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b1c82f0caf42f794759894ad01b850'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.445085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b1c82f0caf42f794759894ad01b850'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.470969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3f079134-9edc-4c40-b28b-c236af3d69be None None] MFA Rules not processed for user `08e4344332a743b4aca2600b1ad4c817`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:53.472685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3f079134-9edc-4c40-b28b-c236af3d69be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:53.479318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b1c82f0caf42f794759894ad01b850'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.500137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:53.523914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1089/2161] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:53 2022] DELETE /identity/v3/users/02b1c82f0caf42f794759894ad01b850 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:53.639772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1073/2162] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:53.646798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:53.649353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:53.695837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f079134-9edc-4c40-b28b-c236af3d69be None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:53.698018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3f079134-9edc-4c40-b28b-c236af3d69be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:53.763743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda63b68-a1af-4b84-a68c-5de0399a10ee admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:53.771895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9417c3-5729-401f-83d5-9a0d31476ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:53.772750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9417c3-5729-401f-83d5-9a0d31476ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:53.773398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9417c3-5729-401f-83d5-9a0d31476ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:53.773695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9417c3-5729-401f-83d5-9a0d31476ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:53.774572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9417c3-5729-401f-83d5-9a0d31476ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:53.778112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9417c3-5729-401f-83d5-9a0d31476ae2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:53.794402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:53.824826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f079134-9edc-4c40-b28b-c236af3d69be None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:53.832037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:53.833213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:53.833491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:53.833786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] PATH_INFO: `/v3/users/b023ac4be4774d22af03d88860a985de` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:53.835345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b023ac4be4774d22af03d88860a985de)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:53.835995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1090/2163] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:53 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:53.840671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:53.841075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9417c3-5729-401f-83d5-9a0d31476ae2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:53.913962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b023ac4be4774d22af03d88860a985de'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.930437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b023ac4be4774d22af03d88860a985de'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.933697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b023ac4be4774d22af03d88860a985de'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.935601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b023ac4be4774d22af03d88860a985de'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.951986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-17d52067-10ff-4328-b2a1-3fa789296a3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b023ac4be4774d22af03d88860a985de'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:53.953407 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9417c3-5729-401f-83d5-9a0d31476ae2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:53.959096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1af21-79ec-4523-8482-5ac3cde21bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:53.959884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1af21-79ec-4523-8482-5ac3cde21bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:53.960247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1af21-79ec-4523-8482-5ac3cde21bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:53.960599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1af21-79ec-4523-8482-5ac3cde21bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:53.961336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1af21-79ec-4523-8482-5ac3cde21bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:53.963516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1af21-79ec-4523-8482-5ac3cde21bc5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:53.997454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1091/2164] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:53 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:53.998548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1af21-79ec-4523-8482-5ac3cde21bc5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:53.998753 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1074/2165] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:53 2022] DELETE /identity/v3/users/b023ac4be4774d22af03d88860a985de => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:54.003893 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-58e1af21-79ec-4523-8482-5ac3cde21bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.005404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edbb0a-6ca8-485b-a4b4-5789531eb6fd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.006249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edbb0a-6ca8-485b-a4b4-5789531eb6fd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.006390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edbb0a-6ca8-485b-a4b4-5789531eb6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.027681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1af21-79ec-4523-8482-5ac3cde21bc5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:54.039876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-68edbb0a-6ca8-485b-a4b4-5789531eb6fd None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:54.041309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-68edbb0a-6ca8-485b-a4b4-5789531eb6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.095583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:54.156326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1075/2166] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:54.157487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edbb0a-6ca8-485b-a4b4-5789531eb6fd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:54.159704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-68edbb0a-6ca8-485b-a4b4-5789531eb6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.177404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edbb0a-6ca8-485b-a4b4-5789531eb6fd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:54.185510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20c177-bdec-4a4d-a53e-8b09160ab488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:54.185687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:54.186549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.186711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20c177-bdec-4a4d-a53e-8b09160ab488 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.186985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.187126 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] PATH_INFO: `/v3/projects/3906b614ecd64109b51f0a634b7a35c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.187274 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20c177-bdec-4a4d-a53e-8b09160ab488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.187704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20c177-bdec-4a4d-a53e-8b09160ab488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.188675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3906b614ecd64109b51f0a634b7a35c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:54.188852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20c177-bdec-4a4d-a53e-8b09160ab488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:54.192379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:54.193114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20c177-bdec-4a4d-a53e-8b09160ab488 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:54.218482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1092/2167] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:54.219786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20c177-bdec-4a4d-a53e-8b09160ab488 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:54.223558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9f20c177-bdec-4a4d-a53e-8b09160ab488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.272407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3906b614ecd64109b51f0a634b7a35c5', 'request_id': 'req-ee8499cf-c18d-40ae-9beb-57208eac2294'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:54.281105 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3906b614ecd64109b51f0a634b7a35c5', 'request_id': 'req-ee8499cf-c18d-40ae-9beb-57208eac2294'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:54.290715 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3906b614ecd64109b51f0a634b7a35c5', 'request_id': 'req-ee8499cf-c18d-40ae-9beb-57208eac2294'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:54.312353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] The token cache is being invalidate because project 3906b614ecd64109b51f0a634b7a35c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:54.312602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:54.313908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:54.317483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1076/2168] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:54 2022] DELETE /identity/v3/projects/3906b614ecd64109b51f0a634b7a35c5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:54.328073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:54.330754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.346900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20c177-bdec-4a4d-a53e-8b09160ab488 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:54.402535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:54.439025 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9a6a9d-f551-4020-87ed-5eccad26b3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:54.439786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a6a9d-f551-4020-87ed-5eccad26b3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.439900 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a6a9d-f551-4020-87ed-5eccad26b3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.440100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9a6a9d-f551-4020-87ed-5eccad26b3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.443072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ca9a6a9d-f551-4020-87ed-5eccad26b3d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.468651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a6a9d-f551-4020-87ed-5eccad26b3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:54.469899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9a6a9d-f551-4020-87ed-5eccad26b3d8 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:54.485439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1093/2169] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:54.488631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8499cf-c18d-40ae-9beb-57208eac2294 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:54.488968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a986-afb3-4490-ada6-38949117caaa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.488968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a986-afb3-4490-ada6-38949117caaa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.488968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6a986-afb3-4490-ada6-38949117caaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.495192 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f72dd-6c11-490a-b975-7b54dbd2f6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:54.495988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f72dd-6c11-490a-b975-7b54dbd2f6bf admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.496520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f72dd-6c11-490a-b975-7b54dbd2f6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.497200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f72dd-6c11-490a-b975-7b54dbd2f6bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.498828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f72dd-6c11-490a-b975-7b54dbd2f6bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:54.500769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f72dd-6c11-490a-b975-7b54dbd2f6bf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:54.515035 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-90d6a986-afb3-4490-ada6-38949117caaa None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:54.517719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-90d6a986-afb3-4490-ada6-38949117caaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.525283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1077/2170] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:54.529613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f72dd-6c11-490a-b975-7b54dbd2f6bf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:54.539305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1094/2171] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:54.540001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6a986-afb3-4490-ada6-38949117caaa None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:54.541818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-90d6a986-afb3-4490-ada6-38949117caaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.553090 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f72dd-6c11-490a-b975-7b54dbd2f6bf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:54.556328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6a986-afb3-4490-ada6-38949117caaa None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:54.559659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1047790-65f6-4fee-811d-c31bd0632388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:54.560581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1047790-65f6-4fee-811d-c31bd0632388 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.561052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1047790-65f6-4fee-811d-c31bd0632388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.561571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1047790-65f6-4fee-811d-c31bd0632388 admin admin] PATH_INFO: `/v3/projects/d1c16435362843e1961747b4fd3b0659/users/15202eba1dfd404a85176ab19a87801e/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.573177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:54.573554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.573873 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.574178 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] PATH_INFO: `/v3/users/f99a876e8f0949f69593767d15be59ec` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.575673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f99a876e8f0949f69593767d15be59ec)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:54.577692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:54.586038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1047790-65f6-4fee-811d-c31bd0632388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c16435362843e1961747b4fd3b0659, user_id=15202eba1dfd404a85176ab19a87801e, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:54.590358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1047790-65f6-4fee-811d-c31bd0632388 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:54.622635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1078/2172] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:54 2022] PUT /identity/v3/projects/d1c16435362843e1961747b4fd3b0659/users/15202eba1dfd404a85176ab19a87801e/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:54.622635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f0295-dc71-4b45-94ec-75965e560012 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.622635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f0295-dc71-4b45-94ec-75965e560012 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.622635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f0295-dc71-4b45-94ec-75965e560012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.648438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9c0f0295-dc71-4b45-94ec-75965e560012 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:54.650804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9c0f0295-dc71-4b45-94ec-75965e560012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.678196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f99a876e8f0949f69593767d15be59ec'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:54.679783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f99a876e8f0949f69593767d15be59ec'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:54.680476 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f99a876e8f0949f69593767d15be59ec'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:54.695030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f99a876e8f0949f69593767d15be59ec'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:54.706651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:54.739381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f99a876e8f0949f69593767d15be59ec'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:54.772878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1079/2173] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:54.774420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff4dcf-a6ea-4e5f-9c89-286a1dbb8d37 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.774707 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff4dcf-a6ea-4e5f-9c89-286a1dbb8d37 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.774979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff4dcf-a6ea-4e5f-9c89-286a1dbb8d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.781947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1095/2174] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:10:54 2022] DELETE /identity/v3/users/f99a876e8f0949f69593767d15be59ec => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:54.783294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:54.826038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-abff4dcf-a6ea-4e5f-9c89-286a1dbb8d37 None None] MFA Rules not processed for user `15202eba1dfd404a85176ab19a87801e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:54.827539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-abff4dcf-a6ea-4e5f-9c89-286a1dbb8d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.905482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13320872-7dd4-45f3-a2d6-9a95db5223a7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:54.912165 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1080/2175] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:54.913889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0394b4-7195-49e9-9279-97ca7556e955 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.914232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0394b4-7195-49e9-9279-97ca7556e955 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.914868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0394b4-7195-49e9-9279-97ca7556e955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.930223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a0fbb-cb10-4f24-a551-04bf1ca75734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:54.934095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a0fbb-cb10-4f24-a551-04bf1ca75734 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:54.934909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a0fbb-cb10-4f24-a551-04bf1ca75734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:54.935792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a0fbb-cb10-4f24-a551-04bf1ca75734 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:54.937438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a0fbb-cb10-4f24-a551-04bf1ca75734 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:10:54.938784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a0fbb-cb10-4f24-a551-04bf1ca75734 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:54.940490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9e0394b4-7195-49e9-9279-97ca7556e955 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:54.942688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9e0394b4-7195-49e9-9279-97ca7556e955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.943593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a0fbb-cb10-4f24-a551-04bf1ca75734 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:54.958128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1081/2176] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:54.960501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0394b4-7195-49e9-9279-97ca7556e955 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:54.972213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9e0394b4-7195-49e9-9279-97ca7556e955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:54.972495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1096/2177] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:10:54.974816 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a0fbb-cb10-4f24-a551-04bf1ca75734 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:54.991971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a0fbb-cb10-4f24-a551-04bf1ca75734 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:54.995403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0394b4-7195-49e9-9279-97ca7556e955 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.000814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b5fc3-0048-45a0-9804-d2217f222d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.006432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b5fc3-0048-45a0-9804-d2217f222d58 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.006999 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b5fc3-0048-45a0-9804-d2217f222d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.007387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b5fc3-0048-45a0-9804-d2217f222d58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.008270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b5fc3-0048-45a0-9804-d2217f222d58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.009300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81310a-225c-4439-9e34-0e767453cee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.010421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81310a-225c-4439-9e34-0e767453cee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.010895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81310a-225c-4439-9e34-0e767453cee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.011376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81310a-225c-4439-9e34-0e767453cee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.012494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81310a-225c-4439-9e34-0e767453cee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.015920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81310a-225c-4439-9e34-0e767453cee7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.020921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b5fc3-0048-45a0-9804-d2217f222d58 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.077970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1082/2178] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:54 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:55.079508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81310a-225c-4439-9e34-0e767453cee7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:55.084264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ee81310a-225c-4439-9e34-0e767453cee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:55.102661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1097/2179] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:54 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:55.104439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b5fc3-0048-45a0-9804-d2217f222d58 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:55.108115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-441b5fc3-0048-45a0-9804-d2217f222d58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:55.124518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:55.195436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81310a-225c-4439-9e34-0e767453cee7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.227886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b5fc3-0048-45a0-9804-d2217f222d58 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.236541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.238461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.239461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.239674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] PATH_INFO: `/v3/projects/c9e00c88dc6e45f48ca271ce9ba5b80a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.241996 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e00c88dc6e45f48ca271ce9ba5b80a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.245011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.268746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:55.275225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025db12-c705-41f2-8a10-e7f02491ba0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.276163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025db12-c705-41f2-8a10-e7f02491ba0a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.276729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025db12-c705-41f2-8a10-e7f02491ba0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.278157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6025db12-c705-41f2-8a10-e7f02491ba0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.292151 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025db12-c705-41f2-8a10-e7f02491ba0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.293712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6025db12-c705-41f2-8a10-e7f02491ba0a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.308173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1083/2180] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:55.309111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025db12-c705-41f2-8a10-e7f02491ba0a service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:55.333049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6025db12-c705-41f2-8a10-e7f02491ba0a service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.340160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e00c88dc6e45f48ca271ce9ba5b80a', 'request_id': 'req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:55.351603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e00c88dc6e45f48ca271ce9ba5b80a', 'request_id': 'req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:55.360644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e00c88dc6e45f48ca271ce9ba5b80a', 'request_id': 'req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:55.380219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] The token cache is being invalidate because project c9e00c88dc6e45f48ca271ce9ba5b80a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:55.380527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:55.381011 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:55.390088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1098/2181] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:55 2022] DELETE /identity/v3/projects/c9e00c88dc6e45f48ca271ce9ba5b80a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:55.390088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:55.390088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:55.520077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1da34-5a73-4e11-900f-e8d0b5e17b2f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.521308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f5813-c6fd-43ec-bbdf-77b5abba88fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.524748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f5813-c6fd-43ec-bbdf-77b5abba88fe admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.527164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f5813-c6fd-43ec-bbdf-77b5abba88fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.527967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f5813-c6fd-43ec-bbdf-77b5abba88fe admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.529755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f5813-c6fd-43ec-bbdf-77b5abba88fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.531693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d626f-e331-44d1-bac0-1c7b7b8c56e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.534012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d626f-e331-44d1-bac0-1c7b7b8c56e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.534522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d626f-e331-44d1-bac0-1c7b7b8c56e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.536514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59d626f-e331-44d1-bac0-1c7b7b8c56e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.538248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d626f-e331-44d1-bac0-1c7b7b8c56e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.541006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f5813-c6fd-43ec-bbdf-77b5abba88fe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.545542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59d626f-e331-44d1-bac0-1c7b7b8c56e9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.593152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1099/2182] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:55 2022] POST /identity/v3/users => generated 287 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:55.602195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d626f-e331-44d1-bac0-1c7b7b8c56e9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:55.620091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1084/2183] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:55 2022] POST /identity/v3/users => generated 294 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:55.625883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f5813-c6fd-43ec-bbdf-77b5abba88fe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:55.640230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5a3f5813-c6fd-43ec-bbdf-77b5abba88fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:55.649082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59d626f-e331-44d1-bac0-1c7b7b8c56e9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.658968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5833-81a2-4de1-ade6-00836418043e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.659276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5833-81a2-4de1-ade6-00836418043e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.660396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5833-81a2-4de1-ade6-00836418043e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.660888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad5833-81a2-4de1-ade6-00836418043e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.662318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad5833-81a2-4de1-ade6-00836418043e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.665346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad5833-81a2-4de1-ade6-00836418043e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.671457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f5813-c6fd-43ec-bbdf-77b5abba88fe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.687937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706840a3-cb78-4f54-a2b5-eb6e5607e62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.691648 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706840a3-cb78-4f54-a2b5-eb6e5607e62e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.692509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706840a3-cb78-4f54-a2b5-eb6e5607e62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.693247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706840a3-cb78-4f54-a2b5-eb6e5607e62e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.694708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706840a3-cb78-4f54-a2b5-eb6e5607e62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.701087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706840a3-cb78-4f54-a2b5-eb6e5607e62e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.713914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1100/2184] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:55.714937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5833-81a2-4de1-ade6-00836418043e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:55.745917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1085/2185] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:55.747347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad5833-81a2-4de1-ade6-00836418043e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.748820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706840a3-cb78-4f54-a2b5-eb6e5607e62e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:55.754652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce1974-fa25-4d39-9306-21f9f70e127a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.756898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1974-fa25-4d39-9306-21f9f70e127a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.756898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1974-fa25-4d39-9306-21f9f70e127a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.756898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1974-fa25-4d39-9306-21f9f70e127a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.758057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce1974-fa25-4d39-9306-21f9f70e127a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.760210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce1974-fa25-4d39-9306-21f9f70e127a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.773469 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706840a3-cb78-4f54-a2b5-eb6e5607e62e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.776367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1101/2186] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:55.784569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce1974-fa25-4d39-9306-21f9f70e127a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:55.793415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a295f24-50e6-420f-b753-ccb51a6653f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.794606 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a295f24-50e6-420f-b753-ccb51a6653f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.795115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a295f24-50e6-420f-b753-ccb51a6653f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.795647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a295f24-50e6-420f-b753-ccb51a6653f1 admin admin] PATH_INFO: `/v3/projects/ce058649e93c4238a58b8327c0fedede/users/8ab542b22f374c3e890e2fe23abf3e86/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.808357 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce1974-fa25-4d39-9306-21f9f70e127a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.821410 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ca96dc-a9e4-4305-881a-dad5609891fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:55.821705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca96dc-a9e4-4305-881a-dad5609891fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.822586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca96dc-a9e4-4305-881a-dad5609891fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.823065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca96dc-a9e4-4305-881a-dad5609891fd admin admin] PATH_INFO: `/v3/projects/2e61a136ef934bf0b994097d12dbb899/users/831e93794b7e429c84bf419c979f0a2b/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.826962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a295f24-50e6-420f-b753-ccb51a6653f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce058649e93c4238a58b8327c0fedede, user_id=8ab542b22f374c3e890e2fe23abf3e86, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.829931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:55.831155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a295f24-50e6-420f-b753-ccb51a6653f1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.850887 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1086/2187] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:55 2022] PUT /identity/v3/projects/ce058649e93c4238a58b8327c0fedede/users/8ab542b22f374c3e890e2fe23abf3e86/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:55.856575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a295f24-50e6-420f-b753-ccb51a6653f1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:55.857279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca96dc-a9e4-4305-881a-dad5609891fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e61a136ef934bf0b994097d12dbb899, user_id=831e93794b7e429c84bf419c979f0a2b, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:55.864332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ca96dc-a9e4-4305-881a-dad5609891fd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:55.897913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1102/2188] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:55 2022] PUT /identity/v3/projects/2e61a136ef934bf0b994097d12dbb899/users/831e93794b7e429c84bf419c979f0a2b/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:55.908640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02825cea-1835-452d-ac54-31c1a10501d4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:55.909254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02825cea-1835-452d-ac54-31c1a10501d4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:55.910688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02825cea-1835-452d-ac54-31c1a10501d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:55.960670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a295f24-50e6-420f-b753-ccb51a6653f1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:55.973020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-02825cea-1835-452d-ac54-31c1a10501d4 None None] MFA Rules not processed for user `831e93794b7e429c84bf419c979f0a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:55.975417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-02825cea-1835-452d-ac54-31c1a10501d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:56.029319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:56.046164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd222b43-711e-4e5e-a8cd-a94d0c3a48d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:56.047076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd222b43-711e-4e5e-a8cd-a94d0c3a48d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.047598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd222b43-711e-4e5e-a8cd-a94d0c3a48d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.048409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd222b43-711e-4e5e-a8cd-a94d0c3a48d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.049264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd222b43-711e-4e5e-a8cd-a94d0c3a48d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:56.058787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd222b43-711e-4e5e-a8cd-a94d0c3a48d2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:56.078133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1087/2189] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:56.078133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd222b43-711e-4e5e-a8cd-a94d0c3a48d2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:56.078133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bd222b43-711e-4e5e-a8cd-a94d0c3a48d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:56.097651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd222b43-711e-4e5e-a8cd-a94d0c3a48d2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:56.106004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:56.106961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.107462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.107890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] PATH_INFO: `/v3/projects/242ec8a2f75a4b0c96829dc4a1f7cb53` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.109283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=242ec8a2f75a4b0c96829dc4a1f7cb53)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:56.112031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:56.193214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1103/2190] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:56.194220 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02825cea-1835-452d-ac54-31c1a10501d4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:56.203569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '242ec8a2f75a4b0c96829dc4a1f7cb53', 'request_id': 'req-6f00abb7-8297-4dba-9ba3-ed148eae0496'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:56.214156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '242ec8a2f75a4b0c96829dc4a1f7cb53', 'request_id': 'req-6f00abb7-8297-4dba-9ba3-ed148eae0496'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:56.223593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '242ec8a2f75a4b0c96829dc4a1f7cb53', 'request_id': 'req-6f00abb7-8297-4dba-9ba3-ed148eae0496'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:56.226769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02825cea-1835-452d-ac54-31c1a10501d4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:56.251630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] The token cache is being invalidate because project 242ec8a2f75a4b0c96829dc4a1f7cb53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:10:56.251630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:56.257688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:10:56.261420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1088/2191] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:10:56 2022] DELETE /identity/v3/projects/242ec8a2f75a4b0c96829dc4a1f7cb53 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:56.263689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:56.271733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:56.342085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb699d-6d2c-441a-b5a7-fa657229aca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:56.343249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb699d-6d2c-441a-b5a7-fa657229aca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.343755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb699d-6d2c-441a-b5a7-fa657229aca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.344416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb699d-6d2c-441a-b5a7-fa657229aca8 admin admin] PATH_INFO: `/v3/projects/ce058649e93c4238a58b8327c0fedede/users/8ab542b22f374c3e890e2fe23abf3e86/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.348861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb699d-6d2c-441a-b5a7-fa657229aca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce058649e93c4238a58b8327c0fedede, user_id=8ab542b22f374c3e890e2fe23abf3e86, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:56.354426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb699d-6d2c-441a-b5a7-fa657229aca8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:56.372754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1104/2192] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:56 2022] PUT /identity/v3/projects/ce058649e93c4238a58b8327c0fedede/users/8ab542b22f374c3e890e2fe23abf3e86/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:56.373838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b1ff7-f021-488c-bfa6-99497cfde31d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.374731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b1ff7-f021-488c-bfa6-99497cfde31d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.375027 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b1ff7-f021-488c-bfa6-99497cfde31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.376911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f00abb7-8297-4dba-9ba3-ed148eae0496 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:56.401576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-790b1ff7-f021-488c-bfa6-99497cfde31d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:56.403260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-790b1ff7-f021-488c-bfa6-99497cfde31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:56.484060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a039b-c9f7-4df0-b180-918263f730f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:56.484969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a039b-c9f7-4df0-b180-918263f730f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.485265 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a039b-c9f7-4df0-b180-918263f730f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.485545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a039b-c9f7-4df0-b180-918263f730f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.486528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a039b-c9f7-4df0-b180-918263f730f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:56.489107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a039b-c9f7-4df0-b180-918263f730f4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:56.506378 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1105/2193] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:56.507945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798db2d1-3798-47dd-a4c5-31deba556b06 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.508234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798db2d1-3798-47dd-a4c5-31deba556b06 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.508504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798db2d1-3798-47dd-a4c5-31deba556b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.538447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:56.538720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1089/2194] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:56 2022] POST /identity/v3/projects => generated 375 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:56.540169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e008287-2a75-4e51-879f-80e9519d08bd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.540169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e008287-2a75-4e51-879f-80e9519d08bd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.540479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e008287-2a75-4e51-879f-80e9519d08bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.554153 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:56.558514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-798db2d1-3798-47dd-a4c5-31deba556b06 None None] MFA Rules not processed for user `8ab542b22f374c3e890e2fe23abf3e86`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:56.560354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-798db2d1-3798-47dd-a4c5-31deba556b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:56.583024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3e008287-2a75-4e51-879f-80e9519d08bd None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:56.585874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3e008287-2a75-4e51-879f-80e9519d08bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:56.672485 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1090/2195] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:56.673970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e008287-2a75-4e51-879f-80e9519d08bd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:56.687769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e008287-2a75-4e51-879f-80e9519d08bd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:56.689139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1106/2196] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:56.690024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798db2d1-3798-47dd-a4c5-31deba556b06 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:56.699888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c82c-4ddb-4ac4-a61e-e952d3cc1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:56.700682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c82c-4ddb-4ac4-a61e-e952d3cc1f04 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.701387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c82c-4ddb-4ac4-a61e-e952d3cc1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.701983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1c82c-4ddb-4ac4-a61e-e952d3cc1f04 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.703692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1c82c-4ddb-4ac4-a61e-e952d3cc1f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:56.707415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a1c82c-4ddb-4ac4-a61e-e952d3cc1f04 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:56.711487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798db2d1-3798-47dd-a4c5-31deba556b06 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:56.716720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adb049-06c7-4bd8-a295-f5958b29761e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:56.717353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb049-06c7-4bd8-a295-f5958b29761e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.718012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb049-06c7-4bd8-a295-f5958b29761e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.718012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb049-06c7-4bd8-a295-f5958b29761e admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.719103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adb049-06c7-4bd8-a295-f5958b29761e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:10:56.719281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adb049-06c7-4bd8-a295-f5958b29761e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:56.720951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adb049-06c7-4bd8-a295-f5958b29761e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:56.738312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1107/2197] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:10:56.738312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adb049-06c7-4bd8-a295-f5958b29761e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:56.738598 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1091/2198] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:56 2022] POST /identity/v3/users => generated 294 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:56.740032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c82c-4ddb-4ac4-a61e-e952d3cc1f04 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:56.743569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b2adb049-06c7-4bd8-a295-f5958b29761e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:56.759363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1c82c-4ddb-4ac4-a61e-e952d3cc1f04 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:56.766085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a8ba9-ac3c-4a2e-89ac-f38765494651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:56.767319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a8ba9-ac3c-4a2e-89ac-f38765494651 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.767559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a8ba9-ac3c-4a2e-89ac-f38765494651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.767832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a8ba9-ac3c-4a2e-89ac-f38765494651 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.768491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a8ba9-ac3c-4a2e-89ac-f38765494651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:56.773557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a8ba9-ac3c-4a2e-89ac-f38765494651 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:56.837312 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1092/2199] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:10:56 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:56.842610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a8ba9-ac3c-4a2e-89ac-f38765494651 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:56.851595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adb049-06c7-4bd8-a295-f5958b29761e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:56.947566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a8ba9-ac3c-4a2e-89ac-f38765494651 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:56.952165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13981d0f-0212-419a-97e7-2f534d609858 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:56.953009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13981d0f-0212-419a-97e7-2f534d609858 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.953838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13981d0f-0212-419a-97e7-2f534d609858 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.954296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13981d0f-0212-419a-97e7-2f534d609858 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.954602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691f8d-b445-4ad9-8f0e-1e42d92c8902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:56.955410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691f8d-b445-4ad9-8f0e-1e42d92c8902 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:56.955794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691f8d-b445-4ad9-8f0e-1e42d92c8902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:56.956193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27691f8d-b445-4ad9-8f0e-1e42d92c8902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:56.957042 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691f8d-b445-4ad9-8f0e-1e42d92c8902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:56.957361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-13981d0f-0212-419a-97e7-2f534d609858 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:56.959054 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27691f8d-b445-4ad9-8f0e-1e42d92c8902 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:56.982243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1093/2200] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:56.984484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691f8d-b445-4ad9-8f0e-1e42d92c8902 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:57.003210 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27691f8d-b445-4ad9-8f0e-1e42d92c8902 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:57.011030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447fc04-7b88-4201-887c-7052f8e55d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:57.012010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447fc04-7b88-4201-887c-7052f8e55d2b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.012382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447fc04-7b88-4201-887c-7052f8e55d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.012731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d447fc04-7b88-4201-887c-7052f8e55d2b admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.013898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447fc04-7b88-4201-887c-7052f8e55d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:57.019543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d447fc04-7b88-4201-887c-7052f8e55d2b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:57.052273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:57.067884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13981d0f-0212-419a-97e7-2f534d609858 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:57.072729 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1094/2201] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:10:56 2022] POST /identity/v3/users => generated 293 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:10:57.073409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13981d0f-0212-419a-97e7-2f534d609858 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:57.073602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447fc04-7b88-4201-887c-7052f8e55d2b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:57.085486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1108/2202] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:10:57.086325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13981d0f-0212-419a-97e7-2f534d609858 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:57.088344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d447fc04-7b88-4201-887c-7052f8e55d2b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:57.099002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c30981-1b91-49f8-9a0a-54afe274a35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:57.100259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c30981-1b91-49f8-9a0a-54afe274a35b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.100907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c30981-1b91-49f8-9a0a-54afe274a35b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.101567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c30981-1b91-49f8-9a0a-54afe274a35b admin admin] PATH_INFO: `/v3/projects/26b24535001f4078bfd53a598310b92d/users/75bd64696f9146e0bffb411d861d6dcf/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.104342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13981d0f-0212-419a-97e7-2f534d609858 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:57.116065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb9ae7-528d-4622-ae16-f4a9bd78b9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:57.117077 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb9ae7-528d-4622-ae16-f4a9bd78b9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.117497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb9ae7-528d-4622-ae16-f4a9bd78b9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.118005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb9ae7-528d-4622-ae16-f4a9bd78b9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.118777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb9ae7-528d-4622-ae16-f4a9bd78b9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:57.120434 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb9ae7-528d-4622-ae16-f4a9bd78b9b6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:57.121659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c30981-1b91-49f8-9a0a-54afe274a35b admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b24535001f4078bfd53a598310b92d, user_id=75bd64696f9146e0bffb411d861d6dcf, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:57.124874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c30981-1b91-49f8-9a0a-54afe274a35b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:57.140919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1109/2203] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:57.147047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1095/2204] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:57 2022] PUT /identity/v3/projects/26b24535001f4078bfd53a598310b92d/users/75bd64696f9146e0bffb411d861d6dcf/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:57.148036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb9ae7-528d-4622-ae16-f4a9bd78b9b6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:57.152737 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c30981-1b91-49f8-9a0a-54afe274a35b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:57.172138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb9ae7-528d-4622-ae16-f4a9bd78b9b6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:57.184688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:57.322024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b16e6-06cd-404b-8cd9-36a68daa3302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:57.322974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b16e6-06cd-404b-8cd9-36a68daa3302 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.323239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b16e6-06cd-404b-8cd9-36a68daa3302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.323520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681b16e6-06cd-404b-8cd9-36a68daa3302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.324289 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b16e6-06cd-404b-8cd9-36a68daa3302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:57.329030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681b16e6-06cd-404b-8cd9-36a68daa3302 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:57.337299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c30981-1b91-49f8-9a0a-54afe274a35b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:57.347843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1e92f-61a6-4e8d-9d0a-bda986504fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:57.348766 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1110/2205] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:10:57.348953 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1e92f-61a6-4e8d-9d0a-bda986504fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.349097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1e92f-61a6-4e8d-9d0a-bda986504fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.349359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1e92f-61a6-4e8d-9d0a-bda986504fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.350066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1e92f-61a6-4e8d-9d0a-bda986504fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:57.352085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1e92f-61a6-4e8d-9d0a-bda986504fd4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:57.355544 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b16e6-06cd-404b-8cd9-36a68daa3302 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:57.365095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1096/2206] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:10:57.369276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1e92f-61a6-4e8d-9d0a-bda986504fd4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:57.370427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681b16e6-06cd-404b-8cd9-36a68daa3302 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:57.380227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5462c-5fa7-42c5-b590-1524cc3fcd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:57.381577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5462c-5fa7-42c5-b590-1524cc3fcd46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.383962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5462c-5fa7-42c5-b590-1524cc3fcd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.384323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5462c-5fa7-42c5-b590-1524cc3fcd46 admin admin] PATH_INFO: `/v3/projects/cf734543b7e0426d93654f18f0d98e4a/users/90472736cbcf497c89f0ae1582bb5ab0/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.387658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1e92f-61a6-4e8d-9d0a-bda986504fd4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:57.400824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd45e-f68b-4dd4-b19f-c6e0199bfee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:57.402408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd45e-f68b-4dd4-b19f-c6e0199bfee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.407125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5462c-5fa7-42c5-b590-1524cc3fcd46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf734543b7e0426d93654f18f0d98e4a, user_id=90472736cbcf497c89f0ae1582bb5ab0, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:57.407501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd45e-f68b-4dd4-b19f-c6e0199bfee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.408122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd45e-f68b-4dd4-b19f-c6e0199bfee1 admin admin] PATH_INFO: `/v3/projects/26b24535001f4078bfd53a598310b92d/users/75bd64696f9146e0bffb411d861d6dcf/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.411470 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5462c-5fa7-42c5-b590-1524cc3fcd46 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:57.416318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd45e-f68b-4dd4-b19f-c6e0199bfee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b24535001f4078bfd53a598310b92d, user_id=75bd64696f9146e0bffb411d861d6dcf, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:57.423735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd45e-f68b-4dd4-b19f-c6e0199bfee1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:57.433236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1111/2207] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:57 2022] PUT /identity/v3/projects/cf734543b7e0426d93654f18f0d98e4a/users/90472736cbcf497c89f0ae1582bb5ab0/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:10:57.438802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3a254-a264-4f3d-8968-4fc93b382b62 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.439171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3a254-a264-4f3d-8968-4fc93b382b62 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.439518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3a254-a264-4f3d-8968-4fc93b382b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.478633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1097/2208] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:10:57 2022] PUT /identity/v3/projects/26b24535001f4078bfd53a598310b92d/users/75bd64696f9146e0bffb411d861d6dcf/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:10:57.485321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d483c5-fe87-4cc1-a21a-c4f92e69d908 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.485852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d483c5-fe87-4cc1-a21a-c4f92e69d908 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.486471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d483c5-fe87-4cc1-a21a-c4f92e69d908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.491753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-00d3a254-a264-4f3d-8968-4fc93b382b62 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:57.494490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-00d3a254-a264-4f3d-8968-4fc93b382b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:57.555169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-06d483c5-fe87-4cc1-a21a-c4f92e69d908 None None] MFA Rules not processed for user `75bd64696f9146e0bffb411d861d6dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:10:57.560662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-06d483c5-fe87-4cc1-a21a-c4f92e69d908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:57.681948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1112/2209] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:10:57.692680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3a254-a264-4f3d-8968-4fc93b382b62 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:57.694462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-00d3a254-a264-4f3d-8968-4fc93b382b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:57.719803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1098/2210] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:10:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:10:57.729313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d483c5-fe87-4cc1-a21a-c4f92e69d908 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:57.788919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3a254-a264-4f3d-8968-4fc93b382b62 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:57.796157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e570581-0d9e-4444-9594-526933e7fe63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:57.797202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e570581-0d9e-4444-9594-526933e7fe63 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.797722 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e570581-0d9e-4444-9594-526933e7fe63 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.798156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e570581-0d9e-4444-9594-526933e7fe63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.807933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d483c5-fe87-4cc1-a21a-c4f92e69d908 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:57.812680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:57.813296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7c6b9-e945-43aa-bc05-97fe158622de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:57.814064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7c6b9-e945-43aa-bc05-97fe158622de service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:57.814580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7c6b9-e945-43aa-bc05-97fe158622de service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:57.814845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a7c6b9-e945-43aa-bc05-97fe158622de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:57.817536 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-67a7c6b9-e945-43aa-bc05-97fe158622de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:57.817848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e570581-0d9e-4444-9594-526933e7fe63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:57.821352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e570581-0d9e-4444-9594-526933e7fe63 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:57.850207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1113/2211] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:10:57.878743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:10:57.927026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7c6b9-e945-43aa-bc05-97fe158622de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:57.928824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a7c6b9-e945-43aa-bc05-97fe158622de service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:57.943998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1099/2212] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:10:58.965111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e570581-0d9e-4444-9594-526933e7fe63 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:10:58.967233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6e570581-0d9e-4444-9594-526933e7fe63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:58.983430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e570581-0d9e-4444-9594-526933e7fe63 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:10:58.989061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bdb0a-9d3d-42ed-85ae-3f7c10769253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:10:58.989847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bdb0a-9d3d-42ed-85ae-3f7c10769253 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:10:58.990294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bdb0a-9d3d-42ed-85ae-3f7c10769253 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:10:58.990576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bdb0a-9d3d-42ed-85ae-3f7c10769253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:10:58.992642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6b4bdb0a-9d3d-42ed-85ae-3f7c10769253 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:10:59.078641 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4bdb0a-9d3d-42ed-85ae-3f7c10769253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:10:59.080279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4bdb0a-9d3d-42ed-85ae-3f7c10769253 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:10:59.092828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1114/2213] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:10:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:11:00.138951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7c6b9-e945-43aa-bc05-97fe158622de service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:00.150217 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a7c6b9-e945-43aa-bc05-97fe158622de service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:00.154612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a4e60-e536-40fb-a21c-0a3e2de842c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:00.155784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a4e60-e536-40fb-a21c-0a3e2de842c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:00.156354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a4e60-e536-40fb-a21c-0a3e2de842c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:00.156818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a4e60-e536-40fb-a21c-0a3e2de842c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:00.237942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a4e60-e536-40fb-a21c-0a3e2de842c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:00.239569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a4e60-e536-40fb-a21c-0a3e2de842c7 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:00.251646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1100/2214] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:11:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:11:00.935904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bdb0a-9d3d-42ed-85ae-3f7c10769253 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:00.963341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bdb0a-9d3d-42ed-85ae-3f7c10769253 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:00.981414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:00.982467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:00.982927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:00.983378 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] PATH_INFO: `/v3/users/831e93794b7e429c84bf419c979f0a2b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:00.984750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=831e93794b7e429c84bf419c979f0a2b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:00.995458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:01.079403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '831e93794b7e429c84bf419c979f0a2b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:01.080168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '831e93794b7e429c84bf419c979f0a2b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:01.081138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '831e93794b7e429c84bf419c979f0a2b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:01.101792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '831e93794b7e429c84bf419c979f0a2b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:01.134150 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '831e93794b7e429c84bf419c979f0a2b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:01.161400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1115/2215] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:11:00 2022] DELETE /identity/v3/users/831e93794b7e429c84bf419c979f0a2b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:01.167273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9d9d-fad6-45ee-94ad-7c8fa0e6ad29 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:01.168177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9d9d-fad6-45ee-94ad-7c8fa0e6ad29 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:01.168690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f9d9d-fad6-45ee-94ad-7c8fa0e6ad29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:01.212992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3e8f9d9d-fad6-45ee-94ad-7c8fa0e6ad29 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:01.216745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3e8f9d9d-fad6-45ee-94ad-7c8fa0e6ad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:01.303043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1101/2216] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:01.315110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:01.317671 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:01.411724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90540994-7f1c-48af-92d3-9098538f2c94 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:01.417455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26942bac-e5d3-435e-b74b-21ce373ac0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:01.418265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942bac-e5d3-435e-b74b-21ce373ac0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:01.418568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942bac-e5d3-435e-b74b-21ce373ac0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:01.418958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26942bac-e5d3-435e-b74b-21ce373ac0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:01.434141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26942bac-e5d3-435e-b74b-21ce373ac0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:01.435990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26942bac-e5d3-435e-b74b-21ce373ac0f9 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:01.449461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1116/2217] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:11:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:11:01.906895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f9d9d-fad6-45ee-94ad-7c8fa0e6ad29 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:01.908400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3e8f9d9d-fad6-45ee-94ad-7c8fa0e6ad29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:01.934487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f9d9d-fad6-45ee-94ad-7c8fa0e6ad29 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:01.953018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:01.953018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:01.953018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:01.953018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] PATH_INFO: `/v3/projects/2e61a136ef934bf0b994097d12dbb899` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:01.959806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e61a136ef934bf0b994097d12dbb899)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:01.964739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:02.026028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e61a136ef934bf0b994097d12dbb899', 'request_id': 'req-96548045-7a30-41cf-b88c-77a0b5616877'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:02.033497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e61a136ef934bf0b994097d12dbb899', 'request_id': 'req-96548045-7a30-41cf-b88c-77a0b5616877'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:02.041594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e61a136ef934bf0b994097d12dbb899', 'request_id': 'req-96548045-7a30-41cf-b88c-77a0b5616877'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:02.059131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:02.064487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] The token cache is being invalidate because project 2e61a136ef934bf0b994097d12dbb899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:11:02.065117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:02.066505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:02.071534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1102/2218] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:11:01 2022] DELETE /identity/v3/projects/2e61a136ef934bf0b994097d12dbb899 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:02.072320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26942bac-e5d3-435e-b74b-21ce373ac0f9 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:02.084558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26942bac-e5d3-435e-b74b-21ce373ac0f9 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:02.094988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:02.191259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:02.192070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:02.192774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:02.192774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] PATH_INFO: `/v3/users/75bd64696f9146e0bffb411d861d6dcf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:02.194898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75bd64696f9146e0bffb411d861d6dcf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:02.197284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:02.288483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75bd64696f9146e0bffb411d861d6dcf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:02.289249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75bd64696f9146e0bffb411d861d6dcf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:02.290957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75bd64696f9146e0bffb411d861d6dcf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:02.300082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75bd64696f9146e0bffb411d861d6dcf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:02.323178 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-da92200a-a22b-4647-a816-2a1c7e728db8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75bd64696f9146e0bffb411d861d6dcf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:02.349934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1117/2219] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:11:02 2022] DELETE /identity/v3/users/75bd64696f9146e0bffb411d861d6dcf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:02.981085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:02.983503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:03.016323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96548045-7a30-41cf-b88c-77a0b5616877 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:03.129178 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:03.130976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:03.131699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:03.132472 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] PATH_INFO: `/v3/projects/26b24535001f4078bfd53a598310b92d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:03.134435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b24535001f4078bfd53a598310b92d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:03.136675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:03.209642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b24535001f4078bfd53a598310b92d', 'request_id': 'req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:03.224362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b24535001f4078bfd53a598310b92d', 'request_id': 'req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:03.231921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b24535001f4078bfd53a598310b92d', 'request_id': 'req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:03.254193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] The token cache is being invalidate because project 26b24535001f4078bfd53a598310b92d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:11:03.254774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:03.255831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-30ece465-64a0-4ff2-aeb2-762f5ac1ffc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:03.262563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1103/2220] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:11:02 2022] DELETE /identity/v3/projects/26b24535001f4078bfd53a598310b92d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:03.264919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c7781-7da2-47cf-9fb9-133003cdefe8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:03.265403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c7781-7da2-47cf-9fb9-133003cdefe8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:03.265488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c7781-7da2-47cf-9fb9-133003cdefe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:03.287523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-079c7781-7da2-47cf-9fb9-133003cdefe8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:03.288927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-079c7781-7da2-47cf-9fb9-133003cdefe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:03.371923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1118/2221] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:03.380541 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7824bb6-0c54-4e63-a8cd-b5822f778e1a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:03.380838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7824bb6-0c54-4e63-a8cd-b5822f778e1a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:03.381117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7824bb6-0c54-4e63-a8cd-b5822f778e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:03.420822 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e7824bb6-0c54-4e63-a8cd-b5822f778e1a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:03.422432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e7824bb6-0c54-4e63-a8cd-b5822f778e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:03.434154 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1104/2222] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:03.439121 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c7781-7da2-47cf-9fb9-133003cdefe8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:03.458468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c7781-7da2-47cf-9fb9-133003cdefe8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:03.464983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8652957-171e-4b04-b9a8-c491bb997727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:03.465691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8652957-171e-4b04-b9a8-c491bb997727 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:03.466081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8652957-171e-4b04-b9a8-c491bb997727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:03.466695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8652957-171e-4b04-b9a8-c491bb997727 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:03.467425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8652957-171e-4b04-b9a8-c491bb997727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:11:03.467809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8652957-171e-4b04-b9a8-c491bb997727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:03.469507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8652957-171e-4b04-b9a8-c491bb997727 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:03.483767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1119/2223] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:11:03.489503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7824bb6-0c54-4e63-a8cd-b5822f778e1a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:03.500339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7824bb6-0c54-4e63-a8cd-b5822f778e1a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:03.511579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cb6d6-a480-49c4-adb0-60ed973b1f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:03.513824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cb6d6-a480-49c4-adb0-60ed973b1f05 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:03.513824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cb6d6-a480-49c4-adb0-60ed973b1f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:03.513824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688cb6d6-a480-49c4-adb0-60ed973b1f05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:03.514561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688cb6d6-a480-49c4-adb0-60ed973b1f05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:03.518970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688cb6d6-a480-49c4-adb0-60ed973b1f05 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:03.564086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8652957-171e-4b04-b9a8-c491bb997727 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:03.573135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1105/2224] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:11:03 2022] POST /identity/v3/projects => generated 365 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:11:03.575835 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8652957-171e-4b04-b9a8-c491bb997727 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:03.590525 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:03.668220 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dcfbf-43e9-4ac6-9461-c83aa6f3b32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:03.668888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dcfbf-43e9-4ac6-9461-c83aa6f3b32a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:03.669367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dcfbf-43e9-4ac6-9461-c83aa6f3b32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:03.669724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3dcfbf-43e9-4ac6-9461-c83aa6f3b32a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:03.670872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3dcfbf-43e9-4ac6-9461-c83aa6f3b32a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:03.673638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3dcfbf-43e9-4ac6-9461-c83aa6f3b32a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:03.713582 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1120/2225] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:11:03 2022] POST /identity/v3/users => generated 283 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:11:03.718963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cb6d6-a480-49c4-adb0-60ed973b1f05 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:03.731623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688cb6d6-a480-49c4-adb0-60ed973b1f05 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:03.740870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bba6b-befb-4a50-988c-2c63137ab9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:03.741600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bba6b-befb-4a50-988c-2c63137ab9cc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:03.741722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bba6b-befb-4a50-988c-2c63137ab9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:03.742205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bba6b-befb-4a50-988c-2c63137ab9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:03.742866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bba6b-befb-4a50-988c-2c63137ab9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:03.744554 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bba6b-befb-4a50-988c-2c63137ab9cc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:03.757129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1106/2226] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:11:03.760821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dcfbf-43e9-4ac6-9461-c83aa6f3b32a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:03.773861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3dcfbf-43e9-4ac6-9461-c83aa6f3b32a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:03.787728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e414bd-ac53-45e7-8f63-94a02bccaecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:03.789910 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e414bd-ac53-45e7-8f63-94a02bccaecb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:03.789910 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e414bd-ac53-45e7-8f63-94a02bccaecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:03.789910 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e414bd-ac53-45e7-8f63-94a02bccaecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:03.802808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e414bd-ac53-45e7-8f63-94a02bccaecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:03.804784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e414bd-ac53-45e7-8f63-94a02bccaecb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:03.821254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1121/2227] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:11:03 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:11:03.825090 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bba6b-befb-4a50-988c-2c63137ab9cc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:03.837380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bba6b-befb-4a50-988c-2c63137ab9cc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:03.842661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502a19f-38bc-4834-85a5-a4f69d509a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:03.843508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502a19f-38bc-4834-85a5-a4f69d509a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:03.843792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502a19f-38bc-4834-85a5-a4f69d509a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:03.844595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502a19f-38bc-4834-85a5-a4f69d509a01 admin admin] PATH_INFO: `/v3/projects/c6511cb1d08543749abe5ce94846217e/users/7ea9216a9fb94fbb8b5de6bdb872d6e2/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:03.870418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502a19f-38bc-4834-85a5-a4f69d509a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6511cb1d08543749abe5ce94846217e, user_id=7ea9216a9fb94fbb8b5de6bdb872d6e2, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:03.873057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502a19f-38bc-4834-85a5-a4f69d509a01 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:03.900070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1107/2228] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:11:03 2022] PUT /identity/v3/projects/c6511cb1d08543749abe5ce94846217e/users/7ea9216a9fb94fbb8b5de6bdb872d6e2/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:11:03.903282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac646f-bf50-4b55-ae67-d676d5379aa5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:03.904007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac646f-bf50-4b55-ae67-d676d5379aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:03.904114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac646f-bf50-4b55-ae67-d676d5379aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:03.954197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a4ac646f-bf50-4b55-ae67-d676d5379aa5 None None] MFA Rules not processed for user `7ea9216a9fb94fbb8b5de6bdb872d6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:03.955658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a4ac646f-bf50-4b55-ae67-d676d5379aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:04.057607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1122/2229] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:11:04.071409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502a19f-38bc-4834-85a5-a4f69d509a01 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:04.078986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6502a19f-38bc-4834-85a5-a4f69d509a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:04.129010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:04.166195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502a19f-38bc-4834-85a5-a4f69d509a01 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:04.173181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491df82-57d8-4cf7-a2cd-2aef24f3c55a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:04.173838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491df82-57d8-4cf7-a2cd-2aef24f3c55a service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:04.174126 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491df82-57d8-4cf7-a2cd-2aef24f3c55a service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:04.174403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491df82-57d8-4cf7-a2cd-2aef24f3c55a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:04.183930 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491df82-57d8-4cf7-a2cd-2aef24f3c55a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:04.185498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7491df82-57d8-4cf7-a2cd-2aef24f3c55a service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:04.197542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1108/2230] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:11:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:11:04.231616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4a1c3-5cd6-49ca-a75e-270f7c77070e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:04.232264 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4a1c3-5cd6-49ca-a75e-270f7c77070e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:04.232751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4a1c3-5cd6-49ca-a75e-270f7c77070e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:04.274598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-52a4a1c3-5cd6-49ca-a75e-270f7c77070e None None] MFA Rules not processed for user `90472736cbcf497c89f0ae1582bb5ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:04.276146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-52a4a1c3-5cd6-49ca-a75e-270f7c77070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:04.369286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1123/2231] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:04.374034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491df82-57d8-4cf7-a2cd-2aef24f3c55a service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:04.375959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7491df82-57d8-4cf7-a2cd-2aef24f3c55a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:04.476670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491df82-57d8-4cf7-a2cd-2aef24f3c55a service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:04.482297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170665a-6414-4a52-9675-e15eb5c549ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:04.483076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170665a-6414-4a52-9675-e15eb5c549ab admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:04.483547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170665a-6414-4a52-9675-e15eb5c549ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:04.483924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5170665a-6414-4a52-9675-e15eb5c549ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:04.484654 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170665a-6414-4a52-9675-e15eb5c549ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:04.488298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5170665a-6414-4a52-9675-e15eb5c549ab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:04.540885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4a1c3-5cd6-49ca-a75e-270f7c77070e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:04.542800 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-52a4a1c3-5cd6-49ca-a75e-270f7c77070e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:04.560695 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1109/2232] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:11:04 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:11:04.562293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170665a-6414-4a52-9675-e15eb5c549ab admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:04.576449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4a1c3-5cd6-49ca-a75e-270f7c77070e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:04.591444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:04.663244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5170665a-6414-4a52-9675-e15eb5c549ab admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:04.668973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec2fc2-0e33-4850-ba20-a43560209d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:04.670978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec2fc2-0e33-4850-ba20-a43560209d5c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:04.670978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec2fc2-0e33-4850-ba20-a43560209d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:04.670978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec2fc2-0e33-4850-ba20-a43560209d5c admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:04.670978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec2fc2-0e33-4850-ba20-a43560209d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:04.673313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ec2fc2-0e33-4850-ba20-a43560209d5c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:04.699272 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:04.699998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:04.700278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:04.700561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] PATH_INFO: `/v3/users/08e4344332a743b4aca2600b1ad4c817` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:04.702256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e4344332a743b4aca2600b1ad4c817)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:04.704438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:04.709212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1110/2233] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:11:04 2022] POST /identity/v3/users => generated 298 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:11:04.710316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec2fc2-0e33-4850-ba20-a43560209d5c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:04.731573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ec2fc2-0e33-4850-ba20-a43560209d5c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:04.744474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ec44d-37cc-485d-84b6-65d6c262aca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:04.744910 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec44d-37cc-485d-84b6-65d6c262aca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:04.745189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec44d-37cc-485d-84b6-65d6c262aca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:04.745535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec44d-37cc-485d-84b6-65d6c262aca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:04.746156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ec44d-37cc-485d-84b6-65d6c262aca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:04.747983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ec44d-37cc-485d-84b6-65d6c262aca6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:04.764909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1111/2234] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:11:04.766807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ec44d-37cc-485d-84b6-65d6c262aca6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:04.785525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ec44d-37cc-485d-84b6-65d6c262aca6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:04.795069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd26d39-a2e0-4581-88f8-0486d0f78208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:04.795763 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd26d39-a2e0-4581-88f8-0486d0f78208 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:04.796041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd26d39-a2e0-4581-88f8-0486d0f78208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:04.796328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd26d39-a2e0-4581-88f8-0486d0f78208 admin admin] PATH_INFO: `/v3/projects/8fc7f40a6149490f9b15ed15e2b15058/users/769e91b4f9c84b0485303a8689924a3e/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:04.796929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e4344332a743b4aca2600b1ad4c817'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:04.798447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e4344332a743b4aca2600b1ad4c817'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:04.799598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e4344332a743b4aca2600b1ad4c817'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:04.815540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e4344332a743b4aca2600b1ad4c817'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:04.816918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd26d39-a2e0-4581-88f8-0486d0f78208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc7f40a6149490f9b15ed15e2b15058, user_id=769e91b4f9c84b0485303a8689924a3e, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:04.820352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd26d39-a2e0-4581-88f8-0486d0f78208 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:04.841812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1112/2235] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:11:04 2022] PUT /identity/v3/projects/8fc7f40a6149490f9b15ed15e2b15058/users/769e91b4f9c84b0485303a8689924a3e/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:04.843540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd26d39-a2e0-4581-88f8-0486d0f78208 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:04.846091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4dd26d39-a2e0-4581-88f8-0486d0f78208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:04.854224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-357191e1-0671-45f8-835f-b319a7f359f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e4344332a743b4aca2600b1ad4c817'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:04.865883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:04.891549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1124/2236] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:11:04 2022] DELETE /identity/v3/users/08e4344332a743b4aca2600b1ad4c817 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:04.914309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31be9b-a345-4338-b0ab-3de83a3d5309 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:04.915373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31be9b-a345-4338-b0ab-3de83a3d5309 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:04.916728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31be9b-a345-4338-b0ab-3de83a3d5309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:04.951504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd26d39-a2e0-4581-88f8-0486d0f78208 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:04.959864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bb31be9b-a345-4338-b0ab-3de83a3d5309 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:04.961097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bb31be9b-a345-4338-b0ab-3de83a3d5309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:05.033129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e2207-b92a-423c-99c6-3ec65576437d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:05.033758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e2207-b92a-423c-99c6-3ec65576437d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:05.034071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e2207-b92a-423c-99c6-3ec65576437d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:05.034994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489e2207-b92a-423c-99c6-3ec65576437d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:05.035857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e2207-b92a-423c-99c6-3ec65576437d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:05.037550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489e2207-b92a-423c-99c6-3ec65576437d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:05.057821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1113/2237] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:11:05.061536 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e2207-b92a-423c-99c6-3ec65576437d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:05.072853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489e2207-b92a-423c-99c6-3ec65576437d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:05.073674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1125/2238] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:05.082951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fea83-3194-4e79-b48a-967671014b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:05.083645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fea83-3194-4e79-b48a-967671014b4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:05.083921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fea83-3194-4e79-b48a-967671014b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:05.084197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246fea83-3194-4e79-b48a-967671014b4f admin admin] PATH_INFO: `/v3/projects/8fc7f40a6149490f9b15ed15e2b15058/users/769e91b4f9c84b0485303a8689924a3e/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:05.086470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fea83-3194-4e79-b48a-967671014b4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fc7f40a6149490f9b15ed15e2b15058, user_id=769e91b4f9c84b0485303a8689924a3e, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:05.090059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246fea83-3194-4e79-b48a-967671014b4f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:05.093615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31be9b-a345-4338-b0ab-3de83a3d5309 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:05.095714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bb31be9b-a345-4338-b0ab-3de83a3d5309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:05.107500 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1114/2239] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:11:05 2022] PUT /identity/v3/projects/8fc7f40a6149490f9b15ed15e2b15058/users/769e91b4f9c84b0485303a8689924a3e/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:11:05.190409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:05.227256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31be9b-a345-4338-b0ab-3de83a3d5309 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:05.295603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f44f74-feac-45e9-8bbc-a97b7b745b70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:05.296270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f44f74-feac-45e9-8bbc-a97b7b745b70 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:05.296534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f44f74-feac-45e9-8bbc-a97b7b745b70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:05.297082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f44f74-feac-45e9-8bbc-a97b7b745b70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:05.370891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f44f74-feac-45e9-8bbc-a97b7b745b70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:05.372508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f44f74-feac-45e9-8bbc-a97b7b745b70 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:05.386832 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1126/2240] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:11:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:11:05.967131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fea83-3194-4e79-b48a-967671014b4f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:05.969698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-246fea83-3194-4e79-b48a-967671014b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:06.011635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246fea83-3194-4e79-b48a-967671014b4f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:06.021784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:06.022894 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:06.023399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:06.023900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] PATH_INFO: `/v3/projects/60c73dd40d27429986f2c3cbd788f9bd` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:06.039346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] RBAC: Authorizing `identity:delete_project(project_id=60c73dd40d27429986f2c3cbd788f9bd)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:06.043254 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:06.207610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60c73dd40d27429986f2c3cbd788f9bd', 'request_id': 'req-1a3975ee-c52b-418b-ad9e-ecb402d0116d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:06.230070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60c73dd40d27429986f2c3cbd788f9bd', 'request_id': 'req-1a3975ee-c52b-418b-ad9e-ecb402d0116d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:06.248102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60c73dd40d27429986f2c3cbd788f9bd', 'request_id': 'req-1a3975ee-c52b-418b-ad9e-ecb402d0116d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:06.280947 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] The token cache is being invalidate because project 60c73dd40d27429986f2c3cbd788f9bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:11:06.281249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:06.282385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:06.288746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1115/2241] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:11:05 2022] DELETE /identity/v3/projects/60c73dd40d27429986f2c3cbd788f9bd => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:06.295997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f44f74-feac-45e9-8bbc-a97b7b745b70 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:06.319144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f44f74-feac-45e9-8bbc-a97b7b745b70 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:06.546327 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:06.551816 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:06.553258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:06.554328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] PATH_INFO: `/v3/users/15202eba1dfd404a85176ab19a87801e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:06.561736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15202eba1dfd404a85176ab19a87801e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:06.566833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:06.709461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:06.727472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15202eba1dfd404a85176ab19a87801e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:06.728536 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15202eba1dfd404a85176ab19a87801e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:06.729628 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15202eba1dfd404a85176ab19a87801e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:06.751537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15202eba1dfd404a85176ab19a87801e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:06.791382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15202eba1dfd404a85176ab19a87801e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:06.811547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1127/2242] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:11:06 2022] DELETE /identity/v3/users/15202eba1dfd404a85176ab19a87801e => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:07.298690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:07.301593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:07.327830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3975ee-c52b-418b-ad9e-ecb402d0116d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:07.393640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:07.500243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:07.501228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:07.501901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:07.502583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] PATH_INFO: `/v3/projects/d1c16435362843e1961747b4fd3b0659` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:07.505226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c16435362843e1961747b4fd3b0659)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:07.508789 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:07.612982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c16435362843e1961747b4fd3b0659', 'request_id': 'req-6dedfa77-9872-4ecd-b755-dca4206d7956'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:07.622919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c16435362843e1961747b4fd3b0659', 'request_id': 'req-6dedfa77-9872-4ecd-b755-dca4206d7956'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:07.632700 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c16435362843e1961747b4fd3b0659', 'request_id': 'req-6dedfa77-9872-4ecd-b755-dca4206d7956'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:07.665941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] The token cache is being invalidate because project d1c16435362843e1961747b4fd3b0659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:11:07.666286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:07.666960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:07.670319 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1116/2243] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:11:07 2022] DELETE /identity/v3/projects/d1c16435362843e1961747b4fd3b0659 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:07.679088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:07.680586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:07.706585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ff5c6-87aa-4ba3-b052-bfa238159604 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:07.868199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:07.868861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:07.869953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:07.870482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] PATH_INFO: `/v3/users/8ab542b22f374c3e890e2fe23abf3e86` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:07.872318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ab542b22f374c3e890e2fe23abf3e86)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:07.874586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:07.974840 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ab542b22f374c3e890e2fe23abf3e86'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:07.975793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ab542b22f374c3e890e2fe23abf3e86'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:07.976706 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ab542b22f374c3e890e2fe23abf3e86'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:08.024885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ab542b22f374c3e890e2fe23abf3e86'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:08.062875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8f09065c-d03a-484c-896c-dc8c4f59dac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ab542b22f374c3e890e2fe23abf3e86'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:08.117775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1128/2244] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:11:07 2022] DELETE /identity/v3/users/8ab542b22f374c3e890e2fe23abf3e86 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:08.734092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:08.735708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:08.759481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedfa77-9872-4ecd-b755-dca4206d7956 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:08.857599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:08.858445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:08.858925 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:08.859279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] PATH_INFO: `/v3/projects/ce058649e93c4238a58b8327c0fedede` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:08.860869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce058649e93c4238a58b8327c0fedede)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:08.863747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:08.937880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce058649e93c4238a58b8327c0fedede', 'request_id': 'req-63337cfc-604b-4160-bbd3-cbaa8109ef79'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:08.954260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce058649e93c4238a58b8327c0fedede', 'request_id': 'req-63337cfc-604b-4160-bbd3-cbaa8109ef79'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:08.961410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:08.965612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce058649e93c4238a58b8327c0fedede', 'request_id': 'req-63337cfc-604b-4160-bbd3-cbaa8109ef79'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:09.000017 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] The token cache is being invalidate because project ce058649e93c4238a58b8327c0fedede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:11:09.000017 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:09.000017 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-63337cfc-604b-4160-bbd3-cbaa8109ef79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:11:09.000867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1117/2245] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:11:08 2022] DELETE /identity/v3/projects/ce058649e93c4238a58b8327c0fedede => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:11:09.006462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21d0b0-d876-43cd-b5dd-030864dcc5cb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:09.007808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21d0b0-d876-43cd-b5dd-030864dcc5cb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:09.008394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e21d0b0-d876-43cd-b5dd-030864dcc5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:09.055680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1e21d0b0-d876-43cd-b5dd-030864dcc5cb None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:09.067491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1e21d0b0-d876-43cd-b5dd-030864dcc5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:09.167236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:09.249597 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9899d8-722b-45dc-aacb-a8944ac2cbf5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:09.250002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9899d8-722b-45dc-aacb-a8944ac2cbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:09.250473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9899d8-722b-45dc-aacb-a8944ac2cbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:09.255929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1129/2246] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:09.287829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9f9899d8-722b-45dc-aacb-a8944ac2cbf5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:09.290132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9f9899d8-722b-45dc-aacb-a8944ac2cbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:09.328678 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1118/2247] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:11:09.330158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21d0b0-d876-43cd-b5dd-030864dcc5cb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:09.350469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e21d0b0-d876-43cd-b5dd-030864dcc5cb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:09.367984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32509493-1955-46d9-a65f-3aa82acfcd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:09.368976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32509493-1955-46d9-a65f-3aa82acfcd33 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:09.369574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32509493-1955-46d9-a65f-3aa82acfcd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:09.370093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32509493-1955-46d9-a65f-3aa82acfcd33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:09.371046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32509493-1955-46d9-a65f-3aa82acfcd33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:11:09.371569 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32509493-1955-46d9-a65f-3aa82acfcd33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:09.373886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32509493-1955-46d9-a65f-3aa82acfcd33 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:09.391727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1130/2248] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:11:09.396470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9899d8-722b-45dc-aacb-a8944ac2cbf5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:09.418632 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9899d8-722b-45dc-aacb-a8944ac2cbf5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:09.433593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191eaca8-cef5-4317-aeca-64286b081fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:09.434386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191eaca8-cef5-4317-aeca-64286b081fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:09.434528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191eaca8-cef5-4317-aeca-64286b081fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:09.434810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191eaca8-cef5-4317-aeca-64286b081fa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:09.435471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191eaca8-cef5-4317-aeca-64286b081fa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:09.442182 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191eaca8-cef5-4317-aeca-64286b081fa9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:09.517524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1119/2249] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:11:09 2022] POST /identity/v3/projects => generated 371 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:11:09.518960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32509493-1955-46d9-a65f-3aa82acfcd33 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:09.536030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32509493-1955-46d9-a65f-3aa82acfcd33 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:09.687071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98f23f-5714-46ce-8f26-8103c700d84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:09.688080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98f23f-5714-46ce-8f26-8103c700d84d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:09.688579 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98f23f-5714-46ce-8f26-8103c700d84d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:09.689084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f98f23f-5714-46ce-8f26-8103c700d84d admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:09.690396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98f23f-5714-46ce-8f26-8103c700d84d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:09.694584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f98f23f-5714-46ce-8f26-8103c700d84d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:09.741859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1131/2250] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:11:09 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:11:09.743089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191eaca8-cef5-4317-aeca-64286b081fa9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:09.760193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191eaca8-cef5-4317-aeca-64286b081fa9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:09.776960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff76a63-e0ba-4d74-8fe5-c03e6cdecacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:09.778289 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff76a63-e0ba-4d74-8fe5-c03e6cdecacf admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:09.778867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff76a63-e0ba-4d74-8fe5-c03e6cdecacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:09.779386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff76a63-e0ba-4d74-8fe5-c03e6cdecacf admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:09.780343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff76a63-e0ba-4d74-8fe5-c03e6cdecacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:09.782338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff76a63-e0ba-4d74-8fe5-c03e6cdecacf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:09.804851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1120/2251] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:11:09.808839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98f23f-5714-46ce-8f26-8103c700d84d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:09.828199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f98f23f-5714-46ce-8f26-8103c700d84d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:09.833452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec27566-053a-4fa7-88a0-b67ec8605286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:09.834139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27566-053a-4fa7-88a0-b67ec8605286 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:09.834425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27566-053a-4fa7-88a0-b67ec8605286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:09.834701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec27566-053a-4fa7-88a0-b67ec8605286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:09.835378 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27566-053a-4fa7-88a0-b67ec8605286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:09.837093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec27566-053a-4fa7-88a0-b67ec8605286 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:09.856399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff76a63-e0ba-4d74-8fe5-c03e6cdecacf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:09.867920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1132/2252] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:11:09.871407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff76a63-e0ba-4d74-8fe5-c03e6cdecacf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:09.912741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2d3da-e00a-4c41-9180-26a75ca41970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:09.921329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2d3da-e00a-4c41-9180-26a75ca41970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:09.923262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2d3da-e00a-4c41-9180-26a75ca41970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:09.923653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2d3da-e00a-4c41-9180-26a75ca41970 admin admin] PATH_INFO: `/v3/projects/c600e063ecf4490087703e6f9c87cefb/users/d0576905fd534e1ead96f5da9a817bb7/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:09.965495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2d3da-e00a-4c41-9180-26a75ca41970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c600e063ecf4490087703e6f9c87cefb, user_id=d0576905fd534e1ead96f5da9a817bb7, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:09.971564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2d3da-e00a-4c41-9180-26a75ca41970 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:10.000731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1121/2253] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:11:09 2022] PUT /identity/v3/projects/c600e063ecf4490087703e6f9c87cefb/users/d0576905fd534e1ead96f5da9a817bb7/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:11:10.007962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74516-8826-4cd7-8eb0-5b615454fa52 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:10.008973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74516-8826-4cd7-8eb0-5b615454fa52 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:10.009482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f74516-8826-4cd7-8eb0-5b615454fa52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:10.042166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-79f74516-8826-4cd7-8eb0-5b615454fa52 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:10.044352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-79f74516-8826-4cd7-8eb0-5b615454fa52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:10.167233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1133/2254] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:10.179179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2d3da-e00a-4c41-9180-26a75ca41970 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:10.181467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-17f2d3da-e00a-4c41-9180-26a75ca41970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:10.318076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2d3da-e00a-4c41-9180-26a75ca41970 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:10.325834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faee3f9-6254-4bd8-b22b-ac4975c565a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:10.327104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faee3f9-6254-4bd8-b22b-ac4975c565a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:10.328840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faee3f9-6254-4bd8-b22b-ac4975c565a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:10.329408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faee3f9-6254-4bd8-b22b-ac4975c565a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:10.355186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faee3f9-6254-4bd8-b22b-ac4975c565a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:10.357105 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faee3f9-6254-4bd8-b22b-ac4975c565a8 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:10.372666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1122/2255] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:11:12.047978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f5f7d-12f6-49ea-846f-f42da93b4fbb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:12.048964 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f5f7d-12f6-49ea-846f-f42da93b4fbb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:12.049465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f5f7d-12f6-49ea-846f-f42da93b4fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:12.088945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-693f5f7d-12f6-49ea-846f-f42da93b4fbb None None] MFA Rules not processed for user `d0576905fd534e1ead96f5da9a817bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:12.091876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-693f5f7d-12f6-49ea-846f-f42da93b4fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:12.140888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:12.236439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1134/2256] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:12.249778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faee3f9-6254-4bd8-b22b-ac4975c565a8 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:12.254024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3faee3f9-6254-4bd8-b22b-ac4975c565a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:12.422613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faee3f9-6254-4bd8-b22b-ac4975c565a8 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:12.432057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca182264-6eb0-4bca-9b3a-ce1d0cfd14d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:12.432610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca182264-6eb0-4bca-9b3a-ce1d0cfd14d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:12.432972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca182264-6eb0-4bca-9b3a-ce1d0cfd14d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:12.433260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca182264-6eb0-4bca-9b3a-ce1d0cfd14d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:12.446602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca182264-6eb0-4bca-9b3a-ce1d0cfd14d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:12.448029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca182264-6eb0-4bca-9b3a-ce1d0cfd14d8 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:12.470133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1123/2257] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:11:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:11:12.938250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f5f7d-12f6-49ea-846f-f42da93b4fbb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:12.943332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-693f5f7d-12f6-49ea-846f-f42da93b4fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:13.152784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f5f7d-12f6-49ea-846f-f42da93b4fbb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:13.160888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc9999-0743-4a8b-809d-3f5fb96a033c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:13.162235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9999-0743-4a8b-809d-3f5fb96a033c service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:13.162728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9999-0743-4a8b-809d-3f5fb96a033c service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:13.163311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc9999-0743-4a8b-809d-3f5fb96a033c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:13.165554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c7dc9999-0743-4a8b-809d-3f5fb96a033c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:13.182283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc9999-0743-4a8b-809d-3f5fb96a033c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:13.185918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc9999-0743-4a8b-809d-3f5fb96a033c service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:13.203974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1135/2258] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:11:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:11:13.434787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb837142-0df2-4de7-8fd0-206488baf515 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:13.435827 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb837142-0df2-4de7-8fd0-206488baf515 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:13.436555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb837142-0df2-4de7-8fd0-206488baf515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:13.489609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-eb837142-0df2-4de7-8fd0-206488baf515 None None] MFA Rules not processed for user `769e91b4f9c84b0485303a8689924a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:13.491187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eb837142-0df2-4de7-8fd0-206488baf515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:13.514274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:13.639653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1124/2259] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:13.647438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc9999-0743-4a8b-809d-3f5fb96a033c service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:13.649064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c7dc9999-0743-4a8b-809d-3f5fb96a033c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:13.728303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc9999-0743-4a8b-809d-3f5fb96a033c service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:13.735597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491680c-0f66-4ed5-86df-141c342a8f35 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b957b367ddf2483f84056ccec57fc515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:13.736234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491680c-0f66-4ed5-86df-141c342a8f35 service glance] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:13.736659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491680c-0f66-4ed5-86df-141c342a8f35 service glance] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:13.736933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491680c-0f66-4ed5-86df-141c342a8f35 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:13.739457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a491680c-0f66-4ed5-86df-141c342a8f35 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:13.827421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491680c-0f66-4ed5-86df-141c342a8f35 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:13.828927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a491680c-0f66-4ed5-86df-141c342a8f35 service glance] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:13.839909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1136/2260] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:11:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:11:13.994667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9649b1-da39-43a6-9714-9c67627b5250 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:13.994947 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9649b1-da39-43a6-9714-9c67627b5250 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:13.996471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9649b1-da39-43a6-9714-9c67627b5250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:14.021032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0d9649b1-da39-43a6-9714-9c67627b5250 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:14.022341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0d9649b1-da39-43a6-9714-9c67627b5250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:14.106329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1125/2261] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:11:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:14.157455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491680c-0f66-4ed5-86df-141c342a8f35 service glance] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:14.160318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a491680c-0f66-4ed5-86df-141c342a8f35 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:14.185969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:14.249546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491680c-0f66-4ed5-86df-141c342a8f35 service glance] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:14.255013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4066d743-31ed-40e7-8d3e-72839d64e5aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:14.255668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066d743-31ed-40e7-8d3e-72839d64e5aa service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:14.255962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066d743-31ed-40e7-8d3e-72839d64e5aa service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:14.256312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066d743-31ed-40e7-8d3e-72839d64e5aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:14.266617 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066d743-31ed-40e7-8d3e-72839d64e5aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:14.269086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4066d743-31ed-40e7-8d3e-72839d64e5aa service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:14.278598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1137/2262] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:11:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:11:18.664304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0dc78-81e4-4b61-8eae-3ea4fc400e34 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:18.664972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0dc78-81e4-4b61-8eae-3ea4fc400e34 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:18.665277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0dc78-81e4-4b61-8eae-3ea4fc400e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:18.708295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-07d0dc78-81e4-4b61-8eae-3ea4fc400e34 None None] MFA Rules not processed for user `d0576905fd534e1ead96f5da9a817bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:18.712780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-07d0dc78-81e4-4b61-8eae-3ea4fc400e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:18.747291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1126/2263] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:24.276005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=24 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:26.797527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462be64-b74a-4dd6-85ac-7f8a8063c92a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:26.798270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462be64-b74a-4dd6-85ac-7f8a8063c92a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:26.798270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462be64-b74a-4dd6-85ac-7f8a8063c92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:26.819669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8462be64-b74a-4dd6-85ac-7f8a8063c92a None None] MFA Rules not processed for user `90472736cbcf497c89f0ae1582bb5ab0`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:26.820951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8462be64-b74a-4dd6-85ac-7f8a8063c92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:26.828099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1138/2264] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:11:28.698377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=51 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:30.284181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585c80c-92ae-4c29-87bd-70273b476311 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:30.284891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585c80c-92ae-4c29-87bd-70273b476311 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:30.284891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585c80c-92ae-4c29-87bd-70273b476311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:30.312925 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0585c80c-92ae-4c29-87bd-70273b476311 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:11:30.314543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0585c80c-92ae-4c29-87bd-70273b476311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:11:30.324457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1127/2265] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:11:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:11:30.342458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462be64-b74a-4dd6-85ac-7f8a8063c92a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:11:30.361594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462be64-b74a-4dd6-85ac-7f8a8063c92a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:11:30.368370 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48d5c5-f229-489a-af97-3e9f35986135 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:11:30.369333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48d5c5-f229-489a-af97-3e9f35986135 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:11:30.369697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48d5c5-f229-489a-af97-3e9f35986135 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:11:30.370019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48d5c5-f229-489a-af97-3e9f35986135 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:11:30.387412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48d5c5-f229-489a-af97-3e9f35986135 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:11:30.389061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c48d5c5-f229-489a-af97-3e9f35986135 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:11:30.402578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1139/2266] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:11:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:11:40.306012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:11:40.396899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=9 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:00.479841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585c80c-92ae-4c29-87bd-70273b476311 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:00.481471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0585c80c-92ae-4c29-87bd-70273b476311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:00.497212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585c80c-92ae-4c29-87bd-70273b476311 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:00.502575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:00.503438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:00.503711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:00.504248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] PATH_INFO: `/v3/users/7ea9216a9fb94fbb8b5de6bdb872d6e2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:00.506840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea9216a9fb94fbb8b5de6bdb872d6e2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:00.510523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:00.576984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea9216a9fb94fbb8b5de6bdb872d6e2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:00.588539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea9216a9fb94fbb8b5de6bdb872d6e2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:00.589185 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea9216a9fb94fbb8b5de6bdb872d6e2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:00.589840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea9216a9fb94fbb8b5de6bdb872d6e2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:00.601325 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea9216a9fb94fbb8b5de6bdb872d6e2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:00.631016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1128/2267] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:12:00 2022] DELETE /identity/v3/users/7ea9216a9fb94fbb8b5de6bdb872d6e2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:12:00.637039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405337d7-2d9b-4aed-b64a-da35fd726d53 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:00.637912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405337d7-2d9b-4aed-b64a-da35fd726d53 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:00.637912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405337d7-2d9b-4aed-b64a-da35fd726d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:00.671079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-405337d7-2d9b-4aed-b64a-da35fd726d53 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:00.671079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-405337d7-2d9b-4aed-b64a-da35fd726d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:00.767784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1140/2268] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:00.780432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:00.781835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:00.858028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af313dfe-c719-4ec7-9dc1-332b3e452340 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:00.862255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e59174-2a96-469b-b674-a444b2379424 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:00.862879 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59174-2a96-469b-b674-a444b2379424 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:00.863158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59174-2a96-469b-b674-a444b2379424 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:00.863448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e59174-2a96-469b-b674-a444b2379424 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:00.875547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e59174-2a96-469b-b674-a444b2379424 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:00.877604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e59174-2a96-469b-b674-a444b2379424 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:00.891471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1129/2269] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:12:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:12:01.201404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405337d7-2d9b-4aed-b64a-da35fd726d53 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:01.202960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-405337d7-2d9b-4aed-b64a-da35fd726d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:01.218652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405337d7-2d9b-4aed-b64a-da35fd726d53 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:01.225045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:01.225715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.226030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.226336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] PATH_INFO: `/v3/projects/c6511cb1d08543749abe5ce94846217e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.227688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6511cb1d08543749abe5ce94846217e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:01.229915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:01.285530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6511cb1d08543749abe5ce94846217e', 'request_id': 'req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:01.295781 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6511cb1d08543749abe5ce94846217e', 'request_id': 'req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:01.303427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6511cb1d08543749abe5ce94846217e', 'request_id': 'req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:01.319443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] The token cache is being invalidate because project c6511cb1d08543749abe5ce94846217e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:12:01.319863 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:01.320683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a4e2af60-f0c3-4765-8e8f-d71c819ec3f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:01.322539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1141/2270] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:12:01 2022] DELETE /identity/v3/projects/c6511cb1d08543749abe5ce94846217e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:12:01.328973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfd76a-7a6e-4796-a674-c4a7dcc5f373 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.329478 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfd76a-7a6e-4796-a674-c4a7dcc5f373 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.329874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bfd76a-7a6e-4796-a674-c4a7dcc5f373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.356896 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e4bfd76a-7a6e-4796-a674-c4a7dcc5f373 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:01.358573 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4bfd76a-7a6e-4796-a674-c4a7dcc5f373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:01.449102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1130/2271] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:12:01.457654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af333f84-063d-4070-a345-d11151a04acb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.458097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af333f84-063d-4070-a345-d11151a04acb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.458333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af333f84-063d-4070-a345-d11151a04acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.493313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-af333f84-063d-4070-a345-d11151a04acb None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:01.495039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-af333f84-063d-4070-a345-d11151a04acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:01.505498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1142/2272] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:01.510532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfd76a-7a6e-4796-a674-c4a7dcc5f373 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:01.523231 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bfd76a-7a6e-4796-a674-c4a7dcc5f373 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:01.528784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6e12c-3d0a-4c01-96a8-26c8502a843b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:01.529423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6e12c-3d0a-4c01-96a8-26c8502a843b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.529750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6e12c-3d0a-4c01-96a8-26c8502a843b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.530063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6e12c-3d0a-4c01-96a8-26c8502a843b admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.530749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6e12c-3d0a-4c01-96a8-26c8502a843b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:12:01.531091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6e12c-3d0a-4c01-96a8-26c8502a843b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:01.533209 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6e12c-3d0a-4c01-96a8-26c8502a843b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:01.549466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1131/2273] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:12:01.555323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af333f84-063d-4070-a345-d11151a04acb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:01.565704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af333f84-063d-4070-a345-d11151a04acb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:01.572578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a26b5-84e8-45bf-a52b-d1f04770fd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:01.573240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a26b5-84e8-45bf-a52b-d1f04770fd4e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.573522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a26b5-84e8-45bf-a52b-d1f04770fd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.573824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346a26b5-84e8-45bf-a52b-d1f04770fd4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.574445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a26b5-84e8-45bf-a52b-d1f04770fd4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:01.576443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346a26b5-84e8-45bf-a52b-d1f04770fd4e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:01.624923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1143/2274] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:12:01 2022] POST /identity/v3/projects => generated 399 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:12:01.631993 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6e12c-3d0a-4c01-96a8-26c8502a843b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:01.638061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:01.648077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6e12c-3d0a-4c01-96a8-26c8502a843b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:01.724141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204818f-f9b2-4f5d-890f-198db6848f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:01.725027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204818f-f9b2-4f5d-890f-198db6848f95 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.725464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204818f-f9b2-4f5d-890f-198db6848f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.725928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204818f-f9b2-4f5d-890f-198db6848f95 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.726998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204818f-f9b2-4f5d-890f-198db6848f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:01.730197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7204818f-f9b2-4f5d-890f-198db6848f95 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:01.756670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1132/2275] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:12:01 2022] POST /identity/v3/users => generated 300 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:12:01.760251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a26b5-84e8-45bf-a52b-d1f04770fd4e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:01.776660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346a26b5-84e8-45bf-a52b-d1f04770fd4e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:01.783409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8702f5-c93d-4374-8869-364ca64ba1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:01.784436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8702f5-c93d-4374-8869-364ca64ba1ba admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.784869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8702f5-c93d-4374-8869-364ca64ba1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.785327 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8702f5-c93d-4374-8869-364ca64ba1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.786818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8702f5-c93d-4374-8869-364ca64ba1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:01.789734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8702f5-c93d-4374-8869-364ca64ba1ba admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:01.817691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1144/2276] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:12:01.820556 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204818f-f9b2-4f5d-890f-198db6848f95 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:01.835532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204818f-f9b2-4f5d-890f-198db6848f95 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:01.843347 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab2c25-d1e5-48bf-b459-62dcc6579cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:01.844893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab2c25-d1e5-48bf-b459-62dcc6579cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.844893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab2c25-d1e5-48bf-b459-62dcc6579cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.845866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab2c25-d1e5-48bf-b459-62dcc6579cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.846814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab2c25-d1e5-48bf-b459-62dcc6579cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:01.850857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab2c25-d1e5-48bf-b459-62dcc6579cf8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:01.862559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1133/2277] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:12:01.868202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8702f5-c93d-4374-8869-364ca64ba1ba admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:01.879164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8702f5-c93d-4374-8869-364ca64ba1ba admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:01.900958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ac9a9-ee7a-45eb-ba22-edc1aa09897f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:01.901663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ac9a9-ee7a-45eb-ba22-edc1aa09897f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.901991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ac9a9-ee7a-45eb-ba22-edc1aa09897f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.902301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ac9a9-ee7a-45eb-ba22-edc1aa09897f admin admin] PATH_INFO: `/v3/projects/a0ecca6e7fec48a98e5e204969b53f85/users/150df97d612b470bb17f3c963e7a0510/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.921730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ac9a9-ee7a-45eb-ba22-edc1aa09897f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ecca6e7fec48a98e5e204969b53f85, user_id=150df97d612b470bb17f3c963e7a0510, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:01.928367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ac9a9-ee7a-45eb-ba22-edc1aa09897f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:01.949108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80e4fe-84a9-4971-b45d-31da6e2cc408 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.949403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80e4fe-84a9-4971-b45d-31da6e2cc408 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.949732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80e4fe-84a9-4971-b45d-31da6e2cc408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.955075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1145/2278] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:12:01 2022] PUT /identity/v3/projects/a0ecca6e7fec48a98e5e204969b53f85/users/150df97d612b470bb17f3c963e7a0510/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:12:01.967439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de522c-7ef8-450c-947d-72ef35ec52c9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:01.967714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de522c-7ef8-450c-947d-72ef35ec52c9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:01.967991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de522c-7ef8-450c-947d-72ef35ec52c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:01.972800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1f80e4fe-84a9-4971-b45d-31da6e2cc408 None None] MFA Rules not processed for user `d0576905fd534e1ead96f5da9a817bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:01.974866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1f80e4fe-84a9-4971-b45d-31da6e2cc408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:02.003269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-11de522c-7ef8-450c-947d-72ef35ec52c9 None None] MFA Rules not processed for user `150df97d612b470bb17f3c963e7a0510`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:02.004916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11de522c-7ef8-450c-947d-72ef35ec52c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:02.015584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:02.073216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1134/2279] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:02.101102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1146/2280] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:02.114621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80e4fe-84a9-4971-b45d-31da6e2cc408 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:02.116179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1f80e4fe-84a9-4971-b45d-31da6e2cc408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:02.204091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80e4fe-84a9-4971-b45d-31da6e2cc408 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:02.210245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06fc55-4b28-4300-95e0-c76698f564e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:02.211345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06fc55-4b28-4300-95e0-c76698f564e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:02.211854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06fc55-4b28-4300-95e0-c76698f564e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:02.212346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06fc55-4b28-4300-95e0-c76698f564e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:02.225344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06fc55-4b28-4300-95e0-c76698f564e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:02.226859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06fc55-4b28-4300-95e0-c76698f564e1 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:02.238380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1135/2281] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:12:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:12:12.094266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=23 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:12.233466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:13.994275 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de522c-7ef8-450c-947d-72ef35ec52c9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:14.188307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de522c-7ef8-450c-947d-72ef35ec52c9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:14.195452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:14.197248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:14.197754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:14.198245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] PATH_INFO: `/v3/users/150df97d612b470bb17f3c963e7a0510` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:14.199729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] RBAC: Authorizing `identity:delete_user(user_id=150df97d612b470bb17f3c963e7a0510)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:14.201807 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:14.334409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '150df97d612b470bb17f3c963e7a0510'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:14.335846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '150df97d612b470bb17f3c963e7a0510'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:14.339589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '150df97d612b470bb17f3c963e7a0510'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:14.358777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '150df97d612b470bb17f3c963e7a0510'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:14.388034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '150df97d612b470bb17f3c963e7a0510'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:14.414063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1147/2282] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:12:13 2022] DELETE /identity/v3/users/150df97d612b470bb17f3c963e7a0510 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:12:14.415911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa8032-30d1-4161-bb6b-6856ce1f7551 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:14.416375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa8032-30d1-4161-bb6b-6856ce1f7551 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:14.416848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaa8032-30d1-4161-bb6b-6856ce1f7551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:14.449446 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9eaa8032-30d1-4161-bb6b-6856ce1f7551 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:14.452003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9eaa8032-30d1-4161-bb6b-6856ce1f7551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:14.605683 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1136/2283] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:14.634162 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:14.636704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:14.806230 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fbf92-05c6-4d45-b118-6c833ad9aaea admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:14.812836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1187f6-36d6-439c-8a7d-6be671306348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:14.814263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1187f6-36d6-439c-8a7d-6be671306348 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:14.814559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1187f6-36d6-439c-8a7d-6be671306348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:14.814904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1187f6-36d6-439c-8a7d-6be671306348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:14.837257 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1187f6-36d6-439c-8a7d-6be671306348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:14.839057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1187f6-36d6-439c-8a7d-6be671306348 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:14.851655 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1148/2284] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:12:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:12:15.266381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa8032-30d1-4161-bb6b-6856ce1f7551 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:15.268658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9eaa8032-30d1-4161-bb6b-6856ce1f7551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:15.296991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaa8032-30d1-4161-bb6b-6856ce1f7551 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:15.309121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:15.310310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:15.310787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:15.311266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] PATH_INFO: `/v3/projects/a0ecca6e7fec48a98e5e204969b53f85` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:15.312841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ecca6e7fec48a98e5e204969b53f85)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:15.318125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:15.386907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ecca6e7fec48a98e5e204969b53f85', 'request_id': 'req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:15.398514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ecca6e7fec48a98e5e204969b53f85', 'request_id': 'req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:15.405188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ecca6e7fec48a98e5e204969b53f85', 'request_id': 'req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:15.421063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] The token cache is being invalidate because project a0ecca6e7fec48a98e5e204969b53f85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:12:15.421688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:15.422029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e907e0e7-ddff-4a7e-bcff-2afb3dc570db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:15.424613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1137/2285] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:12:15 2022] DELETE /identity/v3/projects/a0ecca6e7fec48a98e5e204969b53f85 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:12:23.392821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1187f6-36d6-439c-8a7d-6be671306348 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:23.407963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1187f6-36d6-439c-8a7d-6be671306348 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:23.439268 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:23.486760 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820667e-3ca8-4dfe-ab9c-8ac0f549f112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:23.487907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820667e-3ca8-4dfe-ab9c-8ac0f549f112 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:23.489012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820667e-3ca8-4dfe-ab9c-8ac0f549f112 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:23.489514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820667e-3ca8-4dfe-ab9c-8ac0f549f112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:23.492617 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a820667e-3ca8-4dfe-ab9c-8ac0f549f112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:23.586701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820667e-3ca8-4dfe-ab9c-8ac0f549f112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:23.588740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820667e-3ca8-4dfe-ab9c-8ac0f549f112 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:23.601547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1149/2286] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:12:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:12:25.417498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:31.429857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb06c34-e8e0-4918-9d7b-ae49aa375979 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:31.430793 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb06c34-e8e0-4918-9d7b-ae49aa375979 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:31.431072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb06c34-e8e0-4918-9d7b-ae49aa375979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:31.451443 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9fb06c34-e8e0-4918-9d7b-ae49aa375979 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:31.453067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9fb06c34-e8e0-4918-9d7b-ae49aa375979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:31.513448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1138/2287] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:12:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:31.568573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820667e-3ca8-4dfe-ab9c-8ac0f549f112 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:31.571059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a820667e-3ca8-4dfe-ab9c-8ac0f549f112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:31.656551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820667e-3ca8-4dfe-ab9c-8ac0f549f112 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:31.664751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a3104-d873-442f-9edc-710e99d6dc62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:31.665648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a3104-d873-442f-9edc-710e99d6dc62 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:31.666015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a3104-d873-442f-9edc-710e99d6dc62 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:31.666294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a3104-d873-442f-9edc-710e99d6dc62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:31.679052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a3104-d873-442f-9edc-710e99d6dc62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:31.681450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9a3104-d873-442f-9edc-710e99d6dc62 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:31.695319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1150/2288] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:12:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:12:34.196575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49258f-8e1f-4a67-a52e-fdc4dd8d0659 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:34.197197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49258f-8e1f-4a67-a52e-fdc4dd8d0659 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:34.197197 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e49258f-8e1f-4a67-a52e-fdc4dd8d0659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:34.219906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8e49258f-8e1f-4a67-a52e-fdc4dd8d0659 None None] MFA Rules not processed for user `7f46019e78cc4e89bf877a4f5a7a8fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:34.221248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8e49258f-8e1f-4a67-a52e-fdc4dd8d0659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:34.290802 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1139/2289] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:41.692030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=72 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:44.283463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=50 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:47.248611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a18e83-5c08-4765-b9b2-6573f90a2e0e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:47.249848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a18e83-5c08-4765-b9b2-6573f90a2e0e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:47.250424 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a18e83-5c08-4765-b9b2-6573f90a2e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:47.317528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e4a18e83-5c08-4765-b9b2-6573f90a2e0e None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:47.343617 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e4a18e83-5c08-4765-b9b2-6573f90a2e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:47.357802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1151/2290] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:12:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:47.387489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49258f-8e1f-4a67-a52e-fdc4dd8d0659 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:47.409503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e49258f-8e1f-4a67-a52e-fdc4dd8d0659 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:47.417306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85cafe-2293-4b8d-b69a-b38647a32d80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:47.418089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85cafe-2293-4b8d-b69a-b38647a32d80 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:47.418545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85cafe-2293-4b8d-b69a-b38647a32d80 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:47.418942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a85cafe-2293-4b8d-b69a-b38647a32d80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:47.439156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85cafe-2293-4b8d-b69a-b38647a32d80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:47.443919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a85cafe-2293-4b8d-b69a-b38647a32d80 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:47.462851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1140/2291] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:12:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:12:49.778794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efdb11-c9b3-4ba4-8c6d-e1f68f81c128 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:49.779319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efdb11-c9b3-4ba4-8c6d-e1f68f81c128 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:49.779319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efdb11-c9b3-4ba4-8c6d-e1f68f81c128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:49.801885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-64efdb11-c9b3-4ba4-8c6d-e1f68f81c128 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:49.804859 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-64efdb11-c9b3-4ba4-8c6d-e1f68f81c128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:49.813249 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1152/2292] 10.222.0.16 () {58 vars in 967 bytes} [Fri Aug 19 21:12:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:49.831233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85cafe-2293-4b8d-b69a-b38647a32d80 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:49.842985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a85cafe-2293-4b8d-b69a-b38647a32d80 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:49.848815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e5c5b-1b75-4994-853e-df184f781d4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:49.849423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e5c5b-1b75-4994-853e-df184f781d4d service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:49.849715 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e5c5b-1b75-4994-853e-df184f781d4d service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:49.850031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e5c5b-1b75-4994-853e-df184f781d4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:49.858750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e5c5b-1b75-4994-853e-df184f781d4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:49.860278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e5c5b-1b75-4994-853e-df184f781d4d service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:49.869403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1141/2293] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:12:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:12:52.445156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efdb11-c9b3-4ba4-8c6d-e1f68f81c128 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:52.446779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-64efdb11-c9b3-4ba4-8c6d-e1f68f81c128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:52.564015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efdb11-c9b3-4ba4-8c6d-e1f68f81c128 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:52.575752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:52.577013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:52.577574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:52.578190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] PATH_INFO: `/v3/users/d0576905fd534e1ead96f5da9a817bb7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:52.579948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0576905fd534e1ead96f5da9a817bb7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:52.587489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:52.684286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0576905fd534e1ead96f5da9a817bb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:52.685057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0576905fd534e1ead96f5da9a817bb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:52.685955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0576905fd534e1ead96f5da9a817bb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:52.702743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0576905fd534e1ead96f5da9a817bb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:52.724709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-57d15529-4baa-4442-99db-eedf9966ec00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0576905fd534e1ead96f5da9a817bb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:52.744632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1153/2294] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:12:52 2022] DELETE /identity/v3/users/d0576905fd534e1ead96f5da9a817bb7 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:12:53.339777 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e5c5b-1b75-4994-853e-df184f781d4d service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:53.353734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e5c5b-1b75-4994-853e-df184f781d4d service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:53.506866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:53.508387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:53.510211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:53.512057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] PATH_INFO: `/v3/projects/c600e063ecf4490087703e6f9c87cefb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:53.517503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c600e063ecf4490087703e6f9c87cefb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:53.524351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:53.649671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c600e063ecf4490087703e6f9c87cefb', 'request_id': 'req-c3f3154f-e994-4297-ab57-a15fa5c88aea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:53.663463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c600e063ecf4490087703e6f9c87cefb', 'request_id': 'req-c3f3154f-e994-4297-ab57-a15fa5c88aea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:53.673954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c600e063ecf4490087703e6f9c87cefb', 'request_id': 'req-c3f3154f-e994-4297-ab57-a15fa5c88aea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:53.709058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] The token cache is being invalidate because project c600e063ecf4490087703e6f9c87cefb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:12:53.709972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:53.711070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c3f3154f-e994-4297-ab57-a15fa5c88aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:12:53.713981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1142/2295] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:12:53 2022] DELETE /identity/v3/projects/c600e063ecf4490087703e6f9c87cefb => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:12:53.719732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8125b5-842e-4aff-b1dc-919e81d6cb49 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:53.720601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8125b5-842e-4aff-b1dc-919e81d6cb49 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:53.720601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8125b5-842e-4aff-b1dc-919e81d6cb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:53.757990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4c8125b5-842e-4aff-b1dc-919e81d6cb49 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:53.759554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4c8125b5-842e-4aff-b1dc-919e81d6cb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:53.880110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1154/2296] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:53.889081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3ce7a-0556-4d85-abc4-59833ef6cb69 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:53.889567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3ce7a-0556-4d85-abc4-59833ef6cb69 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:53.890081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3ce7a-0556-4d85-abc4-59833ef6cb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:53.932037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6fc3ce7a-0556-4d85-abc4-59833ef6cb69 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:53.936286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6fc3ce7a-0556-4d85-abc4-59833ef6cb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:53.959918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8125b5-842e-4aff-b1dc-919e81d6cb49 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:53.971008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1143/2297] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:53.978478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8125b5-842e-4aff-b1dc-919e81d6cb49 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:53.990188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f92999-9d2d-42f7-bc05-a42b45d4b851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:53.991718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f92999-9d2d-42f7-bc05-a42b45d4b851 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:53.992304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f92999-9d2d-42f7-bc05-a42b45d4b851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:53.992880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f92999-9d2d-42f7-bc05-a42b45d4b851 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:53.994636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f92999-9d2d-42f7-bc05-a42b45d4b851 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:12:53.995524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f92999-9d2d-42f7-bc05-a42b45d4b851 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:53.998854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f92999-9d2d-42f7-bc05-a42b45d4b851 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:54.002660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:54.037190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1155/2298] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:12:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:12:54.043179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3ce7a-0556-4d85-abc4-59833ef6cb69 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:54.077580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3ce7a-0556-4d85-abc4-59833ef6cb69 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:54.093035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf308b-da6d-4575-97e8-2753c170a4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:54.093690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf308b-da6d-4575-97e8-2753c170a4bf admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:54.093987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf308b-da6d-4575-97e8-2753c170a4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:54.094260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cf308b-da6d-4575-97e8-2753c170a4bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:54.095051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf308b-da6d-4575-97e8-2753c170a4bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:54.098037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cf308b-da6d-4575-97e8-2753c170a4bf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:54.165248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1144/2299] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:12:54 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:12:54.165740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f92999-9d2d-42f7-bc05-a42b45d4b851 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:54.191350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f92999-9d2d-42f7-bc05-a42b45d4b851 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:54.396267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be0e72-79a6-4d5d-88c0-c25607a35f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:54.398911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be0e72-79a6-4d5d-88c0-c25607a35f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:54.399660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be0e72-79a6-4d5d-88c0-c25607a35f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:54.400905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be0e72-79a6-4d5d-88c0-c25607a35f42 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:54.403467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be0e72-79a6-4d5d-88c0-c25607a35f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:54.408412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be0e72-79a6-4d5d-88c0-c25607a35f42 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:54.497355 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf308b-da6d-4575-97e8-2753c170a4bf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:54.498282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1156/2300] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:12:54 2022] POST /identity/v3/users => generated 287 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:12:54.515619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cf308b-da6d-4575-97e8-2753c170a4bf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:54.528291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cec87c-76d1-4926-ae1e-c466e3128c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:54.530398 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cec87c-76d1-4926-ae1e-c466e3128c4b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:54.530847 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cec87c-76d1-4926-ae1e-c466e3128c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:54.531620 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cec87c-76d1-4926-ae1e-c466e3128c4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:54.533988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cec87c-76d1-4926-ae1e-c466e3128c4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:54.540211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cec87c-76d1-4926-ae1e-c466e3128c4b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:54.598028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1145/2301] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:12:54.598260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be0e72-79a6-4d5d-88c0-c25607a35f42 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:54.624820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be0e72-79a6-4d5d-88c0-c25607a35f42 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:54.639381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633a002-e4ba-4400-b58e-bf93c4604778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:54.640164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a002-e4ba-4400-b58e-bf93c4604778 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:54.640452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a002-e4ba-4400-b58e-bf93c4604778 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:54.640731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a002-e4ba-4400-b58e-bf93c4604778 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:54.641624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633a002-e4ba-4400-b58e-bf93c4604778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:54.646045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633a002-e4ba-4400-b58e-bf93c4604778 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:54.676093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1157/2302] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:12:54.677914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cec87c-76d1-4926-ae1e-c466e3128c4b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:54.693819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cec87c-76d1-4926-ae1e-c466e3128c4b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:54.704271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4a0aa-d786-4ca9-8860-c007ca4722f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:54.704957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4a0aa-d786-4ca9-8860-c007ca4722f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:54.705236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4a0aa-d786-4ca9-8860-c007ca4722f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:54.705515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4a0aa-d786-4ca9-8860-c007ca4722f7 admin admin] PATH_INFO: `/v3/projects/7f34422d436a4dbeb7102d599f7996d0/users/b526a58b0c4f49739545419d1fa1feca/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:54.740979 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4a0aa-d786-4ca9-8860-c007ca4722f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f34422d436a4dbeb7102d599f7996d0, user_id=b526a58b0c4f49739545419d1fa1feca, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:54.744158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4a0aa-d786-4ca9-8860-c007ca4722f7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:54.785784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:12:54.799363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1146/2303] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:12:54 2022] PUT /identity/v3/projects/7f34422d436a4dbeb7102d599f7996d0/users/b526a58b0c4f49739545419d1fa1feca/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:12:54.814144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966c94d-b4c2-41ca-a7a5-c227acb9d979 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:54.814740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966c94d-b4c2-41ca-a7a5-c227acb9d979 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:54.814740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966c94d-b4c2-41ca-a7a5-c227acb9d979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:54.859588 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8966c94d-b4c2-41ca-a7a5-c227acb9d979 None None] MFA Rules not processed for user `b526a58b0c4f49739545419d1fa1feca`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:54.861071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8966c94d-b4c2-41ca-a7a5-c227acb9d979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:55.049085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1158/2304] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:12:55.058724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4a0aa-d786-4ca9-8860-c007ca4722f7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:12:55.061449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b9b4a0aa-d786-4ca9-8860-c007ca4722f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:55.161005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4a0aa-d786-4ca9-8860-c007ca4722f7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:12:55.169294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1eb63-edc3-4e34-b1f4-7ec4af0a77a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:12:55.172302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1eb63-edc3-4e34-b1f4-7ec4af0a77a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:55.174222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1eb63-edc3-4e34-b1f4-7ec4af0a77a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:55.175525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1eb63-edc3-4e34-b1f4-7ec4af0a77a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:55.202609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1eb63-edc3-4e34-b1f4-7ec4af0a77a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:12:55.209595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1eb63-edc3-4e34-b1f4-7ec4af0a77a8 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:12:55.227556 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1147/2305] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:12:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:12:56.952154 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921e72d-46a3-4418-956b-7dede0ed61bc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:12:56.952994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921e72d-46a3-4418-956b-7dede0ed61bc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:12:56.953143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921e72d-46a3-4418-956b-7dede0ed61bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:12:56.987571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6921e72d-46a3-4418-956b-7dede0ed61bc None None] MFA Rules not processed for user `b526a58b0c4f49739545419d1fa1feca`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:12:56.990763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6921e72d-46a3-4418-956b-7dede0ed61bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:12:57.006418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1159/2306] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:12:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:13:05.221749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=26 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:13:06.386173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3855-b02b-4b35-9af2-900015c00d16 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:06.386835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3855-b02b-4b35-9af2-900015c00d16 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:06.386835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb3855-b02b-4b35-9af2-900015c00d16 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:06.388196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1148/2307] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:13:06.394812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669a4b2-c5cf-4537-91d4-5c1738ae9385 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:06.395680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669a4b2-c5cf-4537-91d4-5c1738ae9385 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:06.395680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669a4b2-c5cf-4537-91d4-5c1738ae9385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:06.416829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7669a4b2-c5cf-4537-91d4-5c1738ae9385 None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:13:06.418129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7669a4b2-c5cf-4537-91d4-5c1738ae9385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:06.472277 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:13:06.497807 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1160/2308] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:13:06.512623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb3855-b02b-4b35-9af2-900015c00d16 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:06.515012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f1bb3855-b02b-4b35-9af2-900015c00d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:06.577129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb3855-b02b-4b35-9af2-900015c00d16 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:06.580613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db4487-8f22-4cc0-8d0f-b04636cb1636 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b957b367ddf2483f84056ccec57fc515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:06.581914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db4487-8f22-4cc0-8d0f-b04636cb1636 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:06.582165 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db4487-8f22-4cc0-8d0f-b04636cb1636 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:06.582413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db4487-8f22-4cc0-8d0f-b04636cb1636 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:06.593463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db4487-8f22-4cc0-8d0f-b04636cb1636 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:06.594961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0db4487-8f22-4cc0-8d0f-b04636cb1636 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:06.606391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1149/2309] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:13:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:13:06.684180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba3a9-7835-4dc1-b364-f698dcb93f40 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:06.684468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba3a9-7835-4dc1-b364-f698dcb93f40 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:06.684747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeba3a9-7835-4dc1-b364-f698dcb93f40 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:06.686229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1161/2310] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:13:06.691765 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52d591-fc72-438d-833e-32e3e7f64093 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:06.692056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52d591-fc72-438d-833e-32e3e7f64093 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:06.692394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d52d591-fc72-438d-833e-32e3e7f64093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:06.712744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6d52d591-fc72-438d-833e-32e3e7f64093 None None] MFA Rules not processed for user `b37a4275e7c6441aa9e8bb0298f56589`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:13:06.714040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6d52d591-fc72-438d-833e-32e3e7f64093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:06.771675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1150/2311] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:13:06.783909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba3a9-7835-4dc1-b364-f698dcb93f40 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:06.785653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4aeba3a9-7835-4dc1-b364-f698dcb93f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:06.800356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeba3a9-7835-4dc1-b364-f698dcb93f40 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:06.804013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea987c4-f042-4b4f-b2e6-2b351d86477a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:06.804648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea987c4-f042-4b4f-b2e6-2b351d86477a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:06.805044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea987c4-f042-4b4f-b2e6-2b351d86477a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:06.805303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea987c4-f042-4b4f-b2e6-2b351d86477a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:06.812856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea987c4-f042-4b4f-b2e6-2b351d86477a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:06.814337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea987c4-f042-4b4f-b2e6-2b351d86477a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:06.823266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1162/2312] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:13:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:13:06.912469 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d80e7-121b-4098-bdb7-3d3876f90881 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:06.912752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d80e7-121b-4098-bdb7-3d3876f90881 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:06.913034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d80e7-121b-4098-bdb7-3d3876f90881 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:06.914441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1151/2313] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:13:06.920977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52425796-1087-46e9-9238-899298da0123 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:06.921270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52425796-1087-46e9-9238-899298da0123 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:06.921590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52425796-1087-46e9-9238-899298da0123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:06.943305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-52425796-1087-46e9-9238-899298da0123 None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:13:06.944515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-52425796-1087-46e9-9238-899298da0123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:06.953696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1163/2314] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:13:06.964536 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d80e7-121b-4098-bdb7-3d3876f90881 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:06.975663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d80e7-121b-4098-bdb7-3d3876f90881 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:06.979264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65186b6a-cf07-48f7-a354-3b37765ea91c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:06.980146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65186b6a-cf07-48f7-a354-3b37765ea91c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:06.980431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65186b6a-cf07-48f7-a354-3b37765ea91c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:06.980705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65186b6a-cf07-48f7-a354-3b37765ea91c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:06.991969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65186b6a-cf07-48f7-a354-3b37765ea91c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:06.993410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65186b6a-cf07-48f7-a354-3b37765ea91c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:07.005285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1152/2315] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:13:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:13:07.588173 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52425796-1087-46e9-9238-899298da0123 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:07.598339 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52425796-1087-46e9-9238-899298da0123 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:07.601966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6de5f1-e004-4e1c-8f30-9b936c78544a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:07.602563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6de5f1-e004-4e1c-8f30-9b936c78544a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:07.602842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6de5f1-e004-4e1c-8f30-9b936c78544a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:07.603114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6de5f1-e004-4e1c-8f30-9b936c78544a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:07.604845 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eb6de5f1-e004-4e1c-8f30-9b936c78544a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:07.618109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6de5f1-e004-4e1c-8f30-9b936c78544a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:07.619530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6de5f1-e004-4e1c-8f30-9b936c78544a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:07.631491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1164/2316] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:13:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:13:08.389946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65186b6a-cf07-48f7-a354-3b37765ea91c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:08.399290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65186b6a-cf07-48f7-a354-3b37765ea91c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:08.403049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf1270-8371-4c82-a5c5-27718a46ffef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:08.403652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf1270-8371-4c82-a5c5-27718a46ffef service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:08.403928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf1270-8371-4c82-a5c5-27718a46ffef service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:08.404199 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf1270-8371-4c82-a5c5-27718a46ffef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:08.406038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2cbf1270-8371-4c82-a5c5-27718a46ffef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:08.424278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf1270-8371-4c82-a5c5-27718a46ffef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:08.426208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbf1270-8371-4c82-a5c5-27718a46ffef service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:08.437250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1153/2317] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:13:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:13:10.148405 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2767d65-af02-4d53-9c8f-c74f1e537fb9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:10.148878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2767d65-af02-4d53-9c8f-c74f1e537fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:10.148973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2767d65-af02-4d53-9c8f-c74f1e537fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:10.168708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f2767d65-af02-4d53-9c8f-c74f1e537fb9 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:13:10.169973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f2767d65-af02-4d53-9c8f-c74f1e537fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:10.226352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1165/2318] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:13:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:13:10.239131 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf1270-8371-4c82-a5c5-27718a46ffef service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:10.240594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2cbf1270-8371-4c82-a5c5-27718a46ffef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:10.311230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf1270-8371-4c82-a5c5-27718a46ffef service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:10.316890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a847fa-2f82-4a87-b97a-a1b1e5192416 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:10.317498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a847fa-2f82-4a87-b97a-a1b1e5192416 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:10.317785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a847fa-2f82-4a87-b97a-a1b1e5192416 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:10.318072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a847fa-2f82-4a87-b97a-a1b1e5192416 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:10.326761 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a847fa-2f82-4a87-b97a-a1b1e5192416 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:10.328355 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a847fa-2f82-4a87-b97a-a1b1e5192416 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:10.337538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1154/2319] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:13:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:13:20.219534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=46 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:13:20.334312 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:13:29.757957 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2767d65-af02-4d53-9c8f-c74f1e537fb9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:29.759405 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f2767d65-af02-4d53-9c8f-c74f1e537fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:29.773595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2767d65-af02-4d53-9c8f-c74f1e537fb9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:29.777743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01b292-936c-4aea-97b0-06ed687487c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:29.778415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01b292-936c-4aea-97b0-06ed687487c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:29.778694 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01b292-936c-4aea-97b0-06ed687487c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:29.778961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01b292-936c-4aea-97b0-06ed687487c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:29.780849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5e01b292-936c-4aea-97b0-06ed687487c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:29.854708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01b292-936c-4aea-97b0-06ed687487c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:29.856159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01b292-936c-4aea-97b0-06ed687487c3 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:29.866760 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1166/2320] 10.222.0.16 () {60 vars in 1396 bytes} [Fri Aug 19 21:13:29 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:13:38.515074 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bec47-a686-45a3-bb5d-5ed737ea7574 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:38.515607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bec47-a686-45a3-bb5d-5ed737ea7574 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:38.515607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573bec47-a686-45a3-bb5d-5ed737ea7574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:38.536883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-573bec47-a686-45a3-bb5d-5ed737ea7574 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:13:38.538938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-573bec47-a686-45a3-bb5d-5ed737ea7574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:38.547236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1155/2321] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:13:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:13:38.567987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01b292-936c-4aea-97b0-06ed687487c3 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:38.580962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01b292-936c-4aea-97b0-06ed687487c3 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:38.587043 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4276db4-5e22-47b9-808e-8a0f5d68982a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:38.587654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4276db4-5e22-47b9-808e-8a0f5d68982a service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:38.587929 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4276db4-5e22-47b9-808e-8a0f5d68982a service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:38.588197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4276db4-5e22-47b9-808e-8a0f5d68982a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:38.596635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4276db4-5e22-47b9-808e-8a0f5d68982a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:38.598262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4276db4-5e22-47b9-808e-8a0f5d68982a service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:38.606757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1167/2322] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:13:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:13:38.833233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bec47-a686-45a3-bb5d-5ed737ea7574 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:38.850383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573bec47-a686-45a3-bb5d-5ed737ea7574 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:38.853996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f1eeb-3101-47c8-b6e5-df9ca5fb83dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:38.854899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1eeb-3101-47c8-b6e5-df9ca5fb83dd service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:38.855272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1eeb-3101-47c8-b6e5-df9ca5fb83dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:38.855698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1eeb-3101-47c8-b6e5-df9ca5fb83dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:38.857535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-087f1eeb-3101-47c8-b6e5-df9ca5fb83dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:38.871182 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f1eeb-3101-47c8-b6e5-df9ca5fb83dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:38.876662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f1eeb-3101-47c8-b6e5-df9ca5fb83dd service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:38.886473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1156/2323] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:13:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:13:39.207320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ae4e4-b295-499f-9daf-9d76f2eafff2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:39.207608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ae4e4-b295-499f-9daf-9d76f2eafff2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:39.207874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ae4e4-b295-499f-9daf-9d76f2eafff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:39.235187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fb5ae4e4-b295-499f-9daf-9d76f2eafff2 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:13:39.236417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fb5ae4e4-b295-499f-9daf-9d76f2eafff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:39.244748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1168/2324] 10.222.0.16 () {58 vars in 967 bytes} [Fri Aug 19 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:13:39.263480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f1eeb-3101-47c8-b6e5-df9ca5fb83dd service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:39.278341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f1eeb-3101-47c8-b6e5-df9ca5fb83dd service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:39.284029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba8f7e-a52c-45ad-8529-0b3fac986315 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:39.284665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba8f7e-a52c-45ad-8529-0b3fac986315 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:39.284928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba8f7e-a52c-45ad-8529-0b3fac986315 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:39.285206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba8f7e-a52c-45ad-8529-0b3fac986315 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:39.295565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba8f7e-a52c-45ad-8529-0b3fac986315 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:39.297180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba8f7e-a52c-45ad-8529-0b3fac986315 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:39.306677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1157/2325] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:13:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:13:39.552143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752e110-c869-41ff-82d1-dd1f194ecee7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:39.552430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752e110-c869-41ff-82d1-dd1f194ecee7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:39.552726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752e110-c869-41ff-82d1-dd1f194ecee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:39.573670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a752e110-c869-41ff-82d1-dd1f194ecee7 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:13:39.575829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a752e110-c869-41ff-82d1-dd1f194ecee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:39.583942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1169/2326] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:13:39.596697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba8f7e-a52c-45ad-8529-0b3fac986315 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:39.607882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba8f7e-a52c-45ad-8529-0b3fac986315 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:39.612962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f60ecc-f2e4-4e15-a0b3-8496778a06cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:39.613558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f60ecc-f2e4-4e15-a0b3-8496778a06cc service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:39.614010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f60ecc-f2e4-4e15-a0b3-8496778a06cc service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:39.614330 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f60ecc-f2e4-4e15-a0b3-8496778a06cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:39.623661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f60ecc-f2e4-4e15-a0b3-8496778a06cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:39.625237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f60ecc-f2e4-4e15-a0b3-8496778a06cc service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:39.634202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1158/2327] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:13:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:13:39.967181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd3fee-a73d-409d-9f44-f5aab5e91fcf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:39.967459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd3fee-a73d-409d-9f44-f5aab5e91fcf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:39.967748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd3fee-a73d-409d-9f44-f5aab5e91fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:39.987189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-22fd3fee-a73d-409d-9f44-f5aab5e91fcf None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:13:39.988504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-22fd3fee-a73d-409d-9f44-f5aab5e91fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:39.996316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1170/2328] 10.222.0.16 () {58 vars in 967 bytes} [Fri Aug 19 21:13:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:13:40.013115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f60ecc-f2e4-4e15-a0b3-8496778a06cc service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:40.024026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f60ecc-f2e4-4e15-a0b3-8496778a06cc service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:40.031340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b8296-cfaf-431f-9ca5-3c5d31b1390e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:40.032419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b8296-cfaf-431f-9ca5-3c5d31b1390e service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:40.032419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b8296-cfaf-431f-9ca5-3c5d31b1390e service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:40.032623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4b8296-cfaf-431f-9ca5-3c5d31b1390e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:40.042208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b8296-cfaf-431f-9ca5-3c5d31b1390e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:40.043782 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4b8296-cfaf-431f-9ca5-3c5d31b1390e service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:40.052164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1159/2329] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:13:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:13:41.360947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd3fee-a73d-409d-9f44-f5aab5e91fcf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:41.362323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-22fd3fee-a73d-409d-9f44-f5aab5e91fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:41.389135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd3fee-a73d-409d-9f44-f5aab5e91fcf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:41.394719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae4cf0-080e-4510-acf0-854d12e430bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:41.395333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae4cf0-080e-4510-acf0-854d12e430bd service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:41.395613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae4cf0-080e-4510-acf0-854d12e430bd service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:41.395887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae4cf0-080e-4510-acf0-854d12e430bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:41.397669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-76ae4cf0-080e-4510-acf0-854d12e430bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:41.464337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae4cf0-080e-4510-acf0-854d12e430bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:41.465901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae4cf0-080e-4510-acf0-854d12e430bd service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:41.476472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1171/2330] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:13:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:13:42.166535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b8296-cfaf-431f-9ca5-3c5d31b1390e service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:42.230106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4b8296-cfaf-431f-9ca5-3c5d31b1390e service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:42.234876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:42.235497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:42.235777 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:42.236054 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] PATH_INFO: `/v3/users/b526a58b0c4f49739545419d1fa1feca` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:42.237193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b526a58b0c4f49739545419d1fa1feca)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:42.239132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:42.289247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b526a58b0c4f49739545419d1fa1feca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:13:42.296618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b526a58b0c4f49739545419d1fa1feca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:13:42.297295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b526a58b0c4f49739545419d1fa1feca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:13:42.297962 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b526a58b0c4f49739545419d1fa1feca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:13:42.305718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b526a58b0c4f49739545419d1fa1feca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:13:42.322731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1160/2331] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:13:42 2022] DELETE /identity/v3/users/b526a58b0c4f49739545419d1fa1feca => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:13:42.327214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b6d78-0d51-408a-a1bb-a59c12419a79 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:42.327500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b6d78-0d51-408a-a1bb-a59c12419a79 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:42.327775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b6d78-0d51-408a-a1bb-a59c12419a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:42.350153 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-373b6d78-0d51-408a-a1bb-a59c12419a79 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:13:42.350153 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-373b6d78-0d51-408a-a1bb-a59c12419a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:42.407869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1172/2332] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:13:42.418046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:42.419461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:42.441901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:13:42.482236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8a4bc-5abe-432c-a14e-46c541f0f27d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:42.485918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c44d41-580f-4cf7-98aa-620c4c58b95e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:42.486528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c44d41-580f-4cf7-98aa-620c4c58b95e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:42.486810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c44d41-580f-4cf7-98aa-620c4c58b95e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:42.487087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c44d41-580f-4cf7-98aa-620c4c58b95e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:42.497911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c44d41-580f-4cf7-98aa-620c4c58b95e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:42.499362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c44d41-580f-4cf7-98aa-620c4c58b95e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:42.509437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1161/2333] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:13:42.728810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b6d78-0d51-408a-a1bb-a59c12419a79 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:42.730318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-373b6d78-0d51-408a-a1bb-a59c12419a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:42.739831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:13:42.747135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b6d78-0d51-408a-a1bb-a59c12419a79 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:42.754931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:42.756033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:42.756376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:42.756865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] PATH_INFO: `/v3/projects/7f34422d436a4dbeb7102d599f7996d0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:42.758736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f34422d436a4dbeb7102d599f7996d0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:42.761108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:42.814224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f34422d436a4dbeb7102d599f7996d0', 'request_id': 'req-e43d666a-56e0-4961-9b39-7d37e46abe37'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:13:42.822989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f34422d436a4dbeb7102d599f7996d0', 'request_id': 'req-e43d666a-56e0-4961-9b39-7d37e46abe37'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:13:42.829217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f34422d436a4dbeb7102d599f7996d0', 'request_id': 'req-e43d666a-56e0-4961-9b39-7d37e46abe37'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:13:42.841040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] The token cache is being invalidate because project 7f34422d436a4dbeb7102d599f7996d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:13:42.841329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:13:42.842064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:13:42.843734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1173/2334] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:13:42 2022] DELETE /identity/v3/projects/7f34422d436a4dbeb7102d599f7996d0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:13:43.799901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd71a6d-3048-48af-8295-b3b5321a7d6e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:43.800424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd71a6d-3048-48af-8295-b3b5321a7d6e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:43.800424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd71a6d-3048-48af-8295-b3b5321a7d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:43.820785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7dd71a6d-3048-48af-8295-b3b5321a7d6e None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:13:43.822043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7dd71a6d-3048-48af-8295-b3b5321a7d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:43.878419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1162/2335] 10.222.0.16 () {58 vars in 967 bytes} [Fri Aug 19 21:13:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:13:43.891262 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:13:43.892693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:13:43.965229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43d666a-56e0-4961-9b39-7d37e46abe37 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:13:43.970429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594423bd-e4a9-42d0-9f84-45d4a3ab1339 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:13:43.971047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594423bd-e4a9-42d0-9f84-45d4a3ab1339 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:13:43.971331 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594423bd-e4a9-42d0-9f84-45d4a3ab1339 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:13:43.971604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594423bd-e4a9-42d0-9f84-45d4a3ab1339 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:13:43.980585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594423bd-e4a9-42d0-9f84-45d4a3ab1339 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:13:43.982367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594423bd-e4a9-42d0-9f84-45d4a3ab1339 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:13:43.990667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1174/2336] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:13:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:13:53.871633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=42 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:13:53.987151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:14:05.543409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa98048-afd4-4f97-8746-2633ebc5d4dc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:14:05.544767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa98048-afd4-4f97-8746-2633ebc5d4dc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:14:05.544767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa98048-afd4-4f97-8746-2633ebc5d4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:14:05.570783 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0aa98048-afd4-4f97-8746-2633ebc5d4dc None None] MFA Rules not processed for user `e8e345dbc2114b718c98c70844352c63`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:14:05.572377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0aa98048-afd4-4f97-8746-2633ebc5d4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:05.630752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1163/2337] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:14:11.947070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594423bd-e4a9-42d0-9f84-45d4a3ab1339 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:14:11.948543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-594423bd-e4a9-42d0-9f84-45d4a3ab1339 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:12.034317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594423bd-e4a9-42d0-9f84-45d4a3ab1339 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:14:12.040780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980812e-1fda-4302-9723-199aeb0dad85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:14:12.041567 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980812e-1fda-4302-9723-199aeb0dad85 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:14:12.041968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980812e-1fda-4302-9723-199aeb0dad85 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:14:12.042240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980812e-1fda-4302-9723-199aeb0dad85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:14:12.044064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e980812e-1fda-4302-9723-199aeb0dad85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:12.057425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980812e-1fda-4302-9723-199aeb0dad85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:14:12.059115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980812e-1fda-4302-9723-199aeb0dad85 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:14:12.071939 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1175/2338] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:14:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:14:14.847533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa98048-afd4-4f97-8746-2633ebc5d4dc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:14:14.861449 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa98048-afd4-4f97-8746-2633ebc5d4dc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:14:14.869678 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c602c4-e7d5-4956-bde9-1c97980b86e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:14:14.870384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c602c4-e7d5-4956-bde9-1c97980b86e0 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:14:14.870654 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c602c4-e7d5-4956-bde9-1c97980b86e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:14:14.870956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c602c4-e7d5-4956-bde9-1c97980b86e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:14:14.872755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-10c602c4-e7d5-4956-bde9-1c97980b86e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:14.951926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c602c4-e7d5-4956-bde9-1c97980b86e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:14:14.953945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c602c4-e7d5-4956-bde9-1c97980b86e0 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:14:14.968907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1164/2339] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:14:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:14:22.066107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=34 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:14:24.960171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=53 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:14:37.758135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980812e-1fda-4302-9723-199aeb0dad85 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:14:37.769639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980812e-1fda-4302-9723-199aeb0dad85 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:14:37.776049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a8c41-3cea-4dd1-bae9-242c4d434116 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:14:37.776627 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a8c41-3cea-4dd1-bae9-242c4d434116 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:14:37.776899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a8c41-3cea-4dd1-bae9-242c4d434116 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:14:37.777165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a8c41-3cea-4dd1-bae9-242c4d434116 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:14:37.778921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a02a8c41-3cea-4dd1-bae9-242c4d434116 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:37.841493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a8c41-3cea-4dd1-bae9-242c4d434116 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:14:37.843156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a8c41-3cea-4dd1-bae9-242c4d434116 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:14:37.853565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1176/2340] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:14:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:14:41.105882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c602c4-e7d5-4956-bde9-1c97980b86e0 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:14:41.119854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c602c4-e7d5-4956-bde9-1c97980b86e0 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:14:41.126786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b7c50-f62b-4a92-9e0c-8a3573d92d40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:14:41.127445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b7c50-f62b-4a92-9e0c-8a3573d92d40 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:14:41.127701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b7c50-f62b-4a92-9e0c-8a3573d92d40 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:14:41.128053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b7c50-f62b-4a92-9e0c-8a3573d92d40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:14:41.130633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cd0b7c50-f62b-4a92-9e0c-8a3573d92d40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:41.150617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b7c50-f62b-4a92-9e0c-8a3573d92d40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:14:41.152218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b7c50-f62b-4a92-9e0c-8a3573d92d40 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:14:41.165580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1165/2341] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:14:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:14:47.848448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=28 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:14:49.640206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb33c6-c1e2-433f-a22c-79c833ae424c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:14:49.640830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb33c6-c1e2-433f-a22c-79c833ae424c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:14:49.640927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb33c6-c1e2-433f-a22c-79c833ae424c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:14:49.665278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6eeb33c6-c1e2-433f-a22c-79c833ae424c None None] MFA Rules not processed for user `7f46019e78cc4e89bf877a4f5a7a8fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:14:49.665278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6eeb33c6-c1e2-433f-a22c-79c833ae424c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:49.671997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1177/2342] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:14:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:14:49.986320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b7c50-f62b-4a92-9e0c-8a3573d92d40 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:14:49.987824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cd0b7c50-f62b-4a92-9e0c-8a3573d92d40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:50.048795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b7c50-f62b-4a92-9e0c-8a3573d92d40 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:14:50.052429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e0ede-594a-489f-9397-26b7d0eb048e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:14:50.053044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e0ede-594a-489f-9397-26b7d0eb048e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:14:50.053387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e0ede-594a-489f-9397-26b7d0eb048e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:14:50.053754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9e0ede-594a-489f-9397-26b7d0eb048e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:14:50.055487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eb9e0ede-594a-489f-9397-26b7d0eb048e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:50.064820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e0ede-594a-489f-9397-26b7d0eb048e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:14:50.066279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9e0ede-594a-489f-9397-26b7d0eb048e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:14:50.074070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1166/2343] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:14:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:14:50.399047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb33c6-c1e2-433f-a22c-79c833ae424c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:14:50.408280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6eeb33c6-c1e2-433f-a22c-79c833ae424c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:50.480242 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb33c6-c1e2-433f-a22c-79c833ae424c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:14:50.485270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f3cc1-a03b-4cd4-b879-3c416046eb3f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:14:50.485909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f3cc1-a03b-4cd4-b879-3c416046eb3f service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:14:50.486194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f3cc1-a03b-4cd4-b879-3c416046eb3f service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:14:50.486467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f3cc1-a03b-4cd4-b879-3c416046eb3f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:14:50.488207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ec2f3cc1-a03b-4cd4-b879-3c416046eb3f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:50.500137 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f3cc1-a03b-4cd4-b879-3c416046eb3f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:14:50.501778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f3cc1-a03b-4cd4-b879-3c416046eb3f service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:14:50.513589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1178/2344] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:14:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:14:59.630864 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e0ede-594a-489f-9397-26b7d0eb048e service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:14:59.645482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9e0ede-594a-489f-9397-26b7d0eb048e service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:14:59.654596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87c2ca-8b9e-4e62-a666-5095bc87cdfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:14:59.655439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87c2ca-8b9e-4e62-a666-5095bc87cdfa service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:14:59.655778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87c2ca-8b9e-4e62-a666-5095bc87cdfa service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:14:59.656002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87c2ca-8b9e-4e62-a666-5095bc87cdfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:14:59.660634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a87c2ca-8b9e-4e62-a666-5095bc87cdfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:14:59.674949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87c2ca-8b9e-4e62-a666-5095bc87cdfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:14:59.676570 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a87c2ca-8b9e-4e62-a666-5095bc87cdfa service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:14:59.693995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1167/2345] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:14:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:15:00.508693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=35 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:03.627923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04da35a-e104-4534-8814-b9292dbe57ae None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:03.628776 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04da35a-e104-4534-8814-b9292dbe57ae None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:03.628776 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04da35a-e104-4534-8814-b9292dbe57ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:03.652717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a04da35a-e104-4534-8814-b9292dbe57ae None None] MFA Rules not processed for user `0809d96091c2476c83ec0c7c6c5a1509`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:03.653677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a04da35a-e104-4534-8814-b9292dbe57ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:03.736921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1179/2346] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:04.188830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87c2ca-8b9e-4e62-a666-5095bc87cdfa service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:04.200222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87c2ca-8b9e-4e62-a666-5095bc87cdfa service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:04.205323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f536e-fe91-4f22-a8f0-71d7536fa8df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:04.206227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f536e-fe91-4f22-a8f0-71d7536fa8df service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:04.206579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f536e-fe91-4f22-a8f0-71d7536fa8df service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:04.207009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495f536e-fe91-4f22-a8f0-71d7536fa8df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:04.209433 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-495f536e-fe91-4f22-a8f0-71d7536fa8df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:04.225015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f536e-fe91-4f22-a8f0-71d7536fa8df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:04.230955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495f536e-fe91-4f22-a8f0-71d7536fa8df service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:04.243111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1168/2347] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:15:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:15:13.727773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=25 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:14.237227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=44 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:27.659269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953db6a-3858-402f-bbfc-5bd29ba3615b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:27.659987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953db6a-3858-402f-bbfc-5bd29ba3615b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:27.660493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7953db6a-3858-402f-bbfc-5bd29ba3615b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:27.689588 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7953db6a-3858-402f-bbfc-5bd29ba3615b None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:27.690880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7953db6a-3858-402f-bbfc-5bd29ba3615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:27.702103 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1180/2348] 10.222.0.16 () {58 vars in 967 bytes} [Fri Aug 19 21:15:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:15:27.777158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f536e-fe91-4f22-a8f0-71d7536fa8df service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:27.791202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495f536e-fe91-4f22-a8f0-71d7536fa8df service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:27.796670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b6bbe-6a1d-43ab-8438-5f3eb6e735c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:27.797305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b6bbe-6a1d-43ab-8438-5f3eb6e735c5 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:27.797594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b6bbe-6a1d-43ab-8438-5f3eb6e735c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:27.797902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b6bbe-6a1d-43ab-8438-5f3eb6e735c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:27.808296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b6bbe-6a1d-43ab-8438-5f3eb6e735c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:27.810270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290b6bbe-6a1d-43ab-8438-5f3eb6e735c5 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:27.823581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1169/2349] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:15:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:15:32.202140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953db6a-3858-402f-bbfc-5bd29ba3615b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:32.203948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7953db6a-3858-402f-bbfc-5bd29ba3615b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:32.228852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7953db6a-3858-402f-bbfc-5bd29ba3615b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:32.236687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42e0c2-2711-42f0-97bb-190da5e0daaf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:32.237373 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42e0c2-2711-42f0-97bb-190da5e0daaf service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:32.238598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42e0c2-2711-42f0-97bb-190da5e0daaf service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:32.239063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42e0c2-2711-42f0-97bb-190da5e0daaf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:32.242699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0b42e0c2-2711-42f0-97bb-190da5e0daaf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:32.265756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42e0c2-2711-42f0-97bb-190da5e0daaf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:32.267739 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b42e0c2-2711-42f0-97bb-190da5e0daaf service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:32.294186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1181/2350] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:15:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:15:35.514066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b6bbe-6a1d-43ab-8438-5f3eb6e735c5 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:35.526818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290b6bbe-6a1d-43ab-8438-5f3eb6e735c5 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:35.532705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa693b-4805-4948-9519-83346cbdee60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:35.533585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa693b-4805-4948-9519-83346cbdee60 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:35.533827 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa693b-4805-4948-9519-83346cbdee60 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:35.534177 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa693b-4805-4948-9519-83346cbdee60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:35.536025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cafa693b-4805-4948-9519-83346cbdee60 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:35.548810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa693b-4805-4948-9519-83346cbdee60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:35.550572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa693b-4805-4948-9519-83346cbdee60 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:35.576124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1170/2351] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:15:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:15:35.586014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42e0c2-2711-42f0-97bb-190da5e0daaf service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:35.590604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0b42e0c2-2711-42f0-97bb-190da5e0daaf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:35.679001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42e0c2-2711-42f0-97bb-190da5e0daaf service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:35.685904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:35.686974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:35.687474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:35.688032 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] PATH_INFO: `/v3/users/90472736cbcf497c89f0ae1582bb5ab0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:35.689393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=90472736cbcf497c89f0ae1582bb5ab0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:35.691377 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:35.783506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90472736cbcf497c89f0ae1582bb5ab0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:35.784720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90472736cbcf497c89f0ae1582bb5ab0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:35.785562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90472736cbcf497c89f0ae1582bb5ab0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:35.801045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90472736cbcf497c89f0ae1582bb5ab0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:35.821232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90472736cbcf497c89f0ae1582bb5ab0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:35.842183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1182/2352] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:15:35 2022] DELETE /identity/v3/users/90472736cbcf497c89f0ae1582bb5ab0 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:36.035665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa693b-4805-4948-9519-83346cbdee60 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:36.047392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa693b-4805-4948-9519-83346cbdee60 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:36.110228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:36.110870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:36.111170 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:36.111426 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] PATH_INFO: `/v3/projects/cf734543b7e0426d93654f18f0d98e4a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:36.112594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf734543b7e0426d93654f18f0d98e4a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:36.114725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:36.171677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf734543b7e0426d93654f18f0d98e4a', 'request_id': 'req-ddb4747d-0af1-414a-95fc-92504ed1afe6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.183286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf734543b7e0426d93654f18f0d98e4a', 'request_id': 'req-ddb4747d-0af1-414a-95fc-92504ed1afe6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.194077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf734543b7e0426d93654f18f0d98e4a', 'request_id': 'req-ddb4747d-0af1-414a-95fc-92504ed1afe6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.210124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] The token cache is being invalidate because project cf734543b7e0426d93654f18f0d98e4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:15:36.211114 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.212664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.215591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1171/2353] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:15:36 2022] DELETE /identity/v3/projects/cf734543b7e0426d93654f18f0d98e4a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:36.220248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:36.222477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:36.238998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f0c61-a349-4ca8-b80a-493c15d8f4ee admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:36.325592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:36.325592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:36.325592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:36.325592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] PATH_INFO: `/v3/users/769e91b4f9c84b0485303a8689924a3e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:36.326098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=769e91b4f9c84b0485303a8689924a3e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:36.326098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:36.326098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:36.393971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '769e91b4f9c84b0485303a8689924a3e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.394653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '769e91b4f9c84b0485303a8689924a3e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.395271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '769e91b4f9c84b0485303a8689924a3e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.405031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '769e91b4f9c84b0485303a8689924a3e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.421282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ebb78a8d-17de-40e3-af62-fcef6599d3cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '769e91b4f9c84b0485303a8689924a3e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.437811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1183/2354] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:15:36 2022] DELETE /identity/v3/users/769e91b4f9c84b0485303a8689924a3e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:36.647861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:36.649313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:36.664104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4747d-0af1-414a-95fc-92504ed1afe6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:36.729816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:36.730386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:36.730670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:36.730950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] PATH_INFO: `/v3/projects/8fc7f40a6149490f9b15ed15e2b15058` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:36.732346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fc7f40a6149490f9b15ed15e2b15058)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:36.734326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:36.741392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:36.785623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fc7f40a6149490f9b15ed15e2b15058', 'request_id': 'req-c6765276-69af-4c0c-aaba-dd6040078870'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.792948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fc7f40a6149490f9b15ed15e2b15058', 'request_id': 'req-c6765276-69af-4c0c-aaba-dd6040078870'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.799279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fc7f40a6149490f9b15ed15e2b15058', 'request_id': 'req-c6765276-69af-4c0c-aaba-dd6040078870'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.815626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] The token cache is being invalidate because project 8fc7f40a6149490f9b15ed15e2b15058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:15:36.815920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.816549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c6765276-69af-4c0c-aaba-dd6040078870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:36.818102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1172/2355] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:15:36 2022] DELETE /identity/v3/projects/8fc7f40a6149490f9b15ed15e2b15058 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:36.823468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e455a7-855d-4cbe-b4b8-62f2debc2d5f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:36.824040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e455a7-855d-4cbe-b4b8-62f2debc2d5f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:36.824158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e455a7-855d-4cbe-b4b8-62f2debc2d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:36.845382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-63e455a7-855d-4cbe-b4b8-62f2debc2d5f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:36.846613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-63e455a7-855d-4cbe-b4b8-62f2debc2d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:36.911704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1184/2356] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:36.920101 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3d1ff-0678-452e-b097-c52948ffcaef None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:36.920472 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3d1ff-0678-452e-b097-c52948ffcaef None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:36.920820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3d1ff-0678-452e-b097-c52948ffcaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:36.952909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-15f3d1ff-0678-452e-b097-c52948ffcaef None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:36.954193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-15f3d1ff-0678-452e-b097-c52948ffcaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:36.964487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1173/2357] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:36.969782 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e455a7-855d-4cbe-b4b8-62f2debc2d5f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:36.980658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e455a7-855d-4cbe-b4b8-62f2debc2d5f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:36.985275 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63089cee-a8f0-430d-967f-0e92ca23834f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:36.985921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63089cee-a8f0-430d-967f-0e92ca23834f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:36.986201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63089cee-a8f0-430d-967f-0e92ca23834f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:36.986472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63089cee-a8f0-430d-967f-0e92ca23834f admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:36.987125 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63089cee-a8f0-430d-967f-0e92ca23834f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:15:36.987430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63089cee-a8f0-430d-967f-0e92ca23834f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:36.989035 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63089cee-a8f0-430d-967f-0e92ca23834f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:37.002781 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1185/2358] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:15:37.008010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3d1ff-0678-452e-b097-c52948ffcaef None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:37.019345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f3d1ff-0678-452e-b097-c52948ffcaef None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:37.024861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433159f-e9f3-48a1-a4fd-7e0a097b0962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:37.025477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433159f-e9f3-48a1-a4fd-7e0a097b0962 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:37.025751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433159f-e9f3-48a1-a4fd-7e0a097b0962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:37.026047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433159f-e9f3-48a1-a4fd-7e0a097b0962 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:37.026677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433159f-e9f3-48a1-a4fd-7e0a097b0962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:37.028568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433159f-e9f3-48a1-a4fd-7e0a097b0962 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:37.058068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1174/2359] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:15:37 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:15:37.062362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63089cee-a8f0-430d-967f-0e92ca23834f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:37.073255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63089cee-a8f0-430d-967f-0e92ca23834f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:37.135338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd68da-f333-4283-a0ca-a0cb3e68c0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:37.136078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd68da-f333-4283-a0ca-a0cb3e68c0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:37.136363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd68da-f333-4283-a0ca-a0cb3e68c0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:37.136636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd68da-f333-4283-a0ca-a0cb3e68c0f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:37.137540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd68da-f333-4283-a0ca-a0cb3e68c0f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:37.139494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd68da-f333-4283-a0ca-a0cb3e68c0f0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:37.152907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:37.171749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1186/2360] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:15:37 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:15:37.172805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433159f-e9f3-48a1-a4fd-7e0a097b0962 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:37.184266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433159f-e9f3-48a1-a4fd-7e0a097b0962 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:37.189137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef895c69-f473-4403-9fef-77a135c2c9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:37.189746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef895c69-f473-4403-9fef-77a135c2c9ea admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:37.190065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef895c69-f473-4403-9fef-77a135c2c9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:37.190342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef895c69-f473-4403-9fef-77a135c2c9ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:37.191020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef895c69-f473-4403-9fef-77a135c2c9ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:37.192635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef895c69-f473-4403-9fef-77a135c2c9ea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:37.204612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1175/2361] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:37.208672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd68da-f333-4283-a0ca-a0cb3e68c0f0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:37.222705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd68da-f333-4283-a0ca-a0cb3e68c0f0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:37.229271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b9dbd-9f7c-4a7e-a647-f3135ed58e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:37.229932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b9dbd-9f7c-4a7e-a647-f3135ed58e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:37.230214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b9dbd-9f7c-4a7e-a647-f3135ed58e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:37.230494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b9dbd-9f7c-4a7e-a647-f3135ed58e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:37.231198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b9dbd-9f7c-4a7e-a647-f3135ed58e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:37.232884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b9dbd-9f7c-4a7e-a647-f3135ed58e27 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:37.244985 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1187/2362] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:37.249375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef895c69-f473-4403-9fef-77a135c2c9ea admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:37.259688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef895c69-f473-4403-9fef-77a135c2c9ea admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:37.266082 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a31d8b-c0a7-452f-b853-68459dd2b87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:37.266858 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a31d8b-c0a7-452f-b853-68459dd2b87b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:37.267144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a31d8b-c0a7-452f-b853-68459dd2b87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:37.267424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a31d8b-c0a7-452f-b853-68459dd2b87b admin admin] PATH_INFO: `/v3/projects/b8d769840e964ef5923aa9c28265be03/users/f7ed8b3dc88c4fadbb0839fd848397d1/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:37.282781 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a31d8b-c0a7-452f-b853-68459dd2b87b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8d769840e964ef5923aa9c28265be03, user_id=f7ed8b3dc88c4fadbb0839fd848397d1, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:37.285669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a31d8b-c0a7-452f-b853-68459dd2b87b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:37.300626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1176/2363] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:15:37 2022] PUT /identity/v3/projects/b8d769840e964ef5923aa9c28265be03/users/f7ed8b3dc88c4fadbb0839fd848397d1/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:37.307006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6340c09-7500-4cf2-a8c3-97c64d14cb8b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:37.307607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6340c09-7500-4cf2-a8c3-97c64d14cb8b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:37.307983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6340c09-7500-4cf2-a8c3-97c64d14cb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:37.332331 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c6340c09-7500-4cf2-a8c3-97c64d14cb8b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:37.333558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c6340c09-7500-4cf2-a8c3-97c64d14cb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:37.380841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a31d8b-c0a7-452f-b853-68459dd2b87b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:37.382829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c4a31d8b-c0a7-452f-b853-68459dd2b87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:37.422903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1188/2364] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:15:37.434412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6340c09-7500-4cf2-a8c3-97c64d14cb8b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:37.436438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c6340c09-7500-4cf2-a8c3-97c64d14cb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:37.466269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a31d8b-c0a7-452f-b853-68459dd2b87b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:37.471523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484e310-b9b9-4212-a664-57e6f5fe830e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:37.472141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484e310-b9b9-4212-a664-57e6f5fe830e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:37.472421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484e310-b9b9-4212-a664-57e6f5fe830e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:37.472972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b484e310-b9b9-4212-a664-57e6f5fe830e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:37.476032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b484e310-b9b9-4212-a664-57e6f5fe830e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:37.490072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484e310-b9b9-4212-a664-57e6f5fe830e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:37.492087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b484e310-b9b9-4212-a664-57e6f5fe830e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:37.503036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6340c09-7500-4cf2-a8c3-97c64d14cb8b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:37.504600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1177/2365] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:15:37.508023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bb3f6-3c39-48ea-a9fb-fb3684991513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:37.508717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bb3f6-3c39-48ea-a9fb-fb3684991513 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:37.509073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bb3f6-3c39-48ea-a9fb-fb3684991513 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:37.509417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bb3f6-3c39-48ea-a9fb-fb3684991513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:37.519979 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bb3f6-3c39-48ea-a9fb-fb3684991513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:37.522306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5bb3f6-3c39-48ea-a9fb-fb3684991513 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:37.540534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1189/2366] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:15:44.718024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484e310-b9b9-4212-a664-57e6f5fe830e service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:44.719528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b484e310-b9b9-4212-a664-57e6f5fe830e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:44.736071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b484e310-b9b9-4212-a664-57e6f5fe830e service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:44.747511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:44.748392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:44.748838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:44.749813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] PATH_INFO: `/v3/users/e8e345dbc2114b718c98c70844352c63` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:44.750753 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e345dbc2114b718c98c70844352c63)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:44.754818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:44.795497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:44.827558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e345dbc2114b718c98c70844352c63'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:44.836448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e345dbc2114b718c98c70844352c63'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:44.837916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e345dbc2114b718c98c70844352c63'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:44.838855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e345dbc2114b718c98c70844352c63'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:44.859555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef23b6c5-803b-4f9c-bd0c-ff325b95c9fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e345dbc2114b718c98c70844352c63'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:44.895410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1178/2367] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:15:44 2022] DELETE /identity/v3/users/e8e345dbc2114b718c98c70844352c63 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:45.334831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bb3f6-3c39-48ea-a9fb-fb3684991513 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:45.337066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af9933-e940-4c30-bd92-541a9452aea3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:45.337361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af9933-e940-4c30-bd92-541a9452aea3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:45.337702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af9933-e940-4c30-bd92-541a9452aea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:45.346651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bb3f6-3c39-48ea-a9fb-fb3684991513 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:45.404511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b1af9933-e940-4c30-bd92-541a9452aea3 None None] MFA Rules not processed for user `f7ed8b3dc88c4fadbb0839fd848397d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:45.405919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b1af9933-e940-4c30-bd92-541a9452aea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:45.439780 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:45.440381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:45.440622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:45.440945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] PATH_INFO: `/v3/projects/1c4004ba973b4454865b889d6d1e049a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:45.442226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c4004ba973b4454865b889d6d1e049a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:45.445227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:45.517986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1179/2368] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:45.524353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af9933-e940-4c30-bd92-541a9452aea3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:45.526550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b1af9933-e940-4c30-bd92-541a9452aea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:45.529690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:45.553616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af9933-e940-4c30-bd92-541a9452aea3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:45.562547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c4004ba973b4454865b889d6d1e049a', 'request_id': 'req-b060bc6f-dbea-4521-9e01-d8e3af27a045'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:45.590362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c4004ba973b4454865b889d6d1e049a', 'request_id': 'req-b060bc6f-dbea-4521-9e01-d8e3af27a045'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:45.606256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c4004ba973b4454865b889d6d1e049a', 'request_id': 'req-b060bc6f-dbea-4521-9e01-d8e3af27a045'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:45.640087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] The token cache is being invalidate because project 1c4004ba973b4454865b889d6d1e049a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:15:45.640440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:45.644954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b060bc6f-dbea-4521-9e01-d8e3af27a045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:45.645242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf1683-19ab-454f-84b6-5da13dbaa3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:45.645951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf1683-19ab-454f-84b6-5da13dbaa3cd admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:45.647049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf1683-19ab-454f-84b6-5da13dbaa3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:45.647313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf1683-19ab-454f-84b6-5da13dbaa3cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:45.647986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1190/2369] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:15:45 2022] DELETE /identity/v3/projects/1c4004ba973b4454865b889d6d1e049a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:45.650520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf1683-19ab-454f-84b6-5da13dbaa3cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:45.653253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf1683-19ab-454f-84b6-5da13dbaa3cd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:45.653980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850fd43-93ed-40a3-ab4f-1c907c4985d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:45.654397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850fd43-93ed-40a3-ab4f-1c907c4985d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:45.655117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850fd43-93ed-40a3-ab4f-1c907c4985d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:45.702296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1180/2370] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:15:45 2022] POST /identity/v3/projects => generated 371 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:15:45.706438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf1683-19ab-454f-84b6-5da13dbaa3cd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:45.706827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4850fd43-93ed-40a3-ab4f-1c907c4985d5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:45.708115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4850fd43-93ed-40a3-ab4f-1c907c4985d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:45.708328 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1adf1683-19ab-454f-84b6-5da13dbaa3cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:45.745608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:45.816744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf1683-19ab-454f-84b6-5da13dbaa3cd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:45.825019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac2eed-aa86-4e4c-b31a-5fd9a384dcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:45.826668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac2eed-aa86-4e4c-b31a-5fd9a384dcd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:45.826981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac2eed-aa86-4e4c-b31a-5fd9a384dcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:45.827260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac2eed-aa86-4e4c-b31a-5fd9a384dcd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:45.827920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac2eed-aa86-4e4c-b31a-5fd9a384dcd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:45.829974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac2eed-aa86-4e4c-b31a-5fd9a384dcd1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:45.842502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1191/2371] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:45.852375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02326350-4ea6-4cdf-baff-5b6165d90dd2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:45.852684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02326350-4ea6-4cdf-baff-5b6165d90dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:45.852934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02326350-4ea6-4cdf-baff-5b6165d90dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:45.871527 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1181/2372] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:15:45 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:15:45.877044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac2eed-aa86-4e4c-b31a-5fd9a384dcd1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:45.890535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac2eed-aa86-4e4c-b31a-5fd9a384dcd1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:45.893896 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-02326350-4ea6-4cdf-baff-5b6165d90dd2 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:45.895312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-02326350-4ea6-4cdf-baff-5b6165d90dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:45.901248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fef17f-0637-45b1-ae5b-b13d0b45d303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:45.902945 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef17f-0637-45b1-ae5b-b13d0b45d303 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:45.903669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef17f-0637-45b1-ae5b-b13d0b45d303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:45.904637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef17f-0637-45b1-ae5b-b13d0b45d303 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:45.909614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1192/2373] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:15:45.910029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fef17f-0637-45b1-ae5b-b13d0b45d303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:45.910211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02326350-4ea6-4cdf-baff-5b6165d90dd2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:45.917222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fef17f-0637-45b1-ae5b-b13d0b45d303 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:45.925754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02326350-4ea6-4cdf-baff-5b6165d90dd2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:45.934464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80957a2-26dc-449a-9d34-9d6433d39154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:45.935775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80957a2-26dc-449a-9d34-9d6433d39154 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:45.936073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80957a2-26dc-449a-9d34-9d6433d39154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:45.936385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80957a2-26dc-449a-9d34-9d6433d39154 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:45.937640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80957a2-26dc-449a-9d34-9d6433d39154 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:15:45.937640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80957a2-26dc-449a-9d34-9d6433d39154 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:45.940085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80957a2-26dc-449a-9d34-9d6433d39154 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:45.942432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1182/2374] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:45.947853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fef17f-0637-45b1-ae5b-b13d0b45d303 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:45.961828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1193/2375] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:15:45.963878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fef17f-0637-45b1-ae5b-b13d0b45d303 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:45.965185 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80957a2-26dc-449a-9d34-9d6433d39154 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:45.969108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36b5d1-d50a-4e69-bd80-5634081c3690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:45.969839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b5d1-d50a-4e69-bd80-5634081c3690 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:45.970135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b5d1-d50a-4e69-bd80-5634081c3690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:45.971085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b5d1-d50a-4e69-bd80-5634081c3690 admin admin] PATH_INFO: `/v3/projects/3b027eba284e4e6a9d6b018c23a0fd10/users/b329612022194c9dacf15ec93814fcfd/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:45.982925 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80957a2-26dc-449a-9d34-9d6433d39154 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:45.993629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36b5d1-d50a-4e69-bd80-5634081c3690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b027eba284e4e6a9d6b018c23a0fd10, user_id=b329612022194c9dacf15ec93814fcfd, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:45.994815 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd41dc-e244-44d0-b9ac-c36caddc7fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:45.995899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd41dc-e244-44d0-b9ac-c36caddc7fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:45.996398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd41dc-e244-44d0-b9ac-c36caddc7fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:45.996882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd41dc-e244-44d0-b9ac-c36caddc7fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:45.998300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd41dc-e244-44d0-b9ac-c36caddc7fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:45.998428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36b5d1-d50a-4e69-bd80-5634081c3690 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:46.002438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd41dc-e244-44d0-b9ac-c36caddc7fe6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:46.023084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1183/2376] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:15:45 2022] PUT /identity/v3/projects/3b027eba284e4e6a9d6b018c23a0fd10/users/b329612022194c9dacf15ec93814fcfd/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:46.031897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36b5d1-d50a-4e69-bd80-5634081c3690 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:46.057020 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1194/2377] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:15:45 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:15:46.062475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd41dc-e244-44d0-b9ac-c36caddc7fe6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:46.133697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36b5d1-d50a-4e69-bd80-5634081c3690 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:46.231404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd41dc-e244-44d0-b9ac-c36caddc7fe6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:46.240524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941b9f0-d97a-400d-9b53-8577beec64f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:46.241608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941b9f0-d97a-400d-9b53-8577beec64f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:46.242191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941b9f0-d97a-400d-9b53-8577beec64f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:46.242737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941b9f0-d97a-400d-9b53-8577beec64f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:46.244241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941b9f0-d97a-400d-9b53-8577beec64f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:46.244539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638ea56-9999-415d-ae72-6d22c7c2c714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:46.245593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638ea56-9999-415d-ae72-6d22c7c2c714 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:46.246165 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638ea56-9999-415d-ae72-6d22c7c2c714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:46.246580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638ea56-9999-415d-ae72-6d22c7c2c714 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:46.247839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638ea56-9999-415d-ae72-6d22c7c2c714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:46.248246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941b9f0-d97a-400d-9b53-8577beec64f5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:46.250697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638ea56-9999-415d-ae72-6d22c7c2c714 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:46.274662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1184/2378] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:46.274662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97efc28-782b-412f-9fbb-e1eb742d7398 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:46.274662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97efc28-782b-412f-9fbb-e1eb742d7398 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:46.274662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97efc28-782b-412f-9fbb-e1eb742d7398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:46.294911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1195/2379] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:15:46 2022] POST /identity/v3/users => generated 287 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:15:46.294911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941b9f0-d97a-400d-9b53-8577beec64f5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:46.306989 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b97efc28-782b-412f-9fbb-e1eb742d7398 None None] MFA Rules not processed for user `8810b3e211d448cea9a506d9183dc13d`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:46.309041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b97efc28-782b-412f-9fbb-e1eb742d7398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:46.314826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941b9f0-d97a-400d-9b53-8577beec64f5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:46.325282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d25b2d-460e-448e-8da5-9ef6b48be6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:46.326775 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d25b2d-460e-448e-8da5-9ef6b48be6b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:46.327550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d25b2d-460e-448e-8da5-9ef6b48be6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:46.328289 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d25b2d-460e-448e-8da5-9ef6b48be6b5 admin admin] PATH_INFO: `/v3/projects/3b027eba284e4e6a9d6b018c23a0fd10/users/b329612022194c9dacf15ec93814fcfd/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:46.330574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:46.334794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d25b2d-460e-448e-8da5-9ef6b48be6b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b027eba284e4e6a9d6b018c23a0fd10, user_id=b329612022194c9dacf15ec93814fcfd, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:46.343065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d25b2d-460e-448e-8da5-9ef6b48be6b5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:46.350634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:46.371382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1196/2380] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:15:46 2022] PUT /identity/v3/projects/3b027eba284e4e6a9d6b018c23a0fd10/users/b329612022194c9dacf15ec93814fcfd/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:46.385595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d25b2d-460e-448e-8da5-9ef6b48be6b5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:46.433418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1185/2381] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:46.493380 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d25b2d-460e-448e-8da5-9ef6b48be6b5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:46.500303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa194eb-f0d5-4f43-b371-1b67721e1444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:46.502324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa194eb-f0d5-4f43-b371-1b67721e1444 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:46.502802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa194eb-f0d5-4f43-b371-1b67721e1444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:46.503087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa194eb-f0d5-4f43-b371-1b67721e1444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:46.503922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa194eb-f0d5-4f43-b371-1b67721e1444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:46.508488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa194eb-f0d5-4f43-b371-1b67721e1444 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:46.531729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1197/2382] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:46.533736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97efc28-782b-412f-9fbb-e1eb742d7398 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:46.551079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97efc28-782b-412f-9fbb-e1eb742d7398 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:46.558327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dafb2-55da-4a44-9a38-ff942457bf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:46.558992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dafb2-55da-4a44-9a38-ff942457bf7f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:46.559274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dafb2-55da-4a44-9a38-ff942457bf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:46.562520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dafb2-55da-4a44-9a38-ff942457bf7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:46.563800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dafb2-55da-4a44-9a38-ff942457bf7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:46.567835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dafb2-55da-4a44-9a38-ff942457bf7f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:46.591731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1186/2383] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:46.596718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa194eb-f0d5-4f43-b371-1b67721e1444 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:46.614467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa194eb-f0d5-4f43-b371-1b67721e1444 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:46.623751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05498bae-4764-486b-8010-40f42bcd74ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:46.624915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05498bae-4764-486b-8010-40f42bcd74ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:46.625636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05498bae-4764-486b-8010-40f42bcd74ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:46.626173 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05498bae-4764-486b-8010-40f42bcd74ca admin admin] PATH_INFO: `/v3/projects/2e364e5252464d30a002dc479c82c062/users/d13425bfe5f74ef0b8237033ec946baf/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:46.654703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05498bae-4764-486b-8010-40f42bcd74ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e364e5252464d30a002dc479c82c062, user_id=d13425bfe5f74ef0b8237033ec946baf, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:46.659225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05498bae-4764-486b-8010-40f42bcd74ca admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:46.686998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1198/2384] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:15:46 2022] PUT /identity/v3/projects/2e364e5252464d30a002dc479c82c062/users/d13425bfe5f74ef0b8237033ec946baf/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:15:46.690206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18902138-b1c7-4266-a1ad-650a57a54133 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:46.690423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18902138-b1c7-4266-a1ad-650a57a54133 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:46.690723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18902138-b1c7-4266-a1ad-650a57a54133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:46.732461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-18902138-b1c7-4266-a1ad-650a57a54133 None None] MFA Rules not processed for user `d13425bfe5f74ef0b8237033ec946baf`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:46.734656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-18902138-b1c7-4266-a1ad-650a57a54133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:46.826482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1187/2385] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:15:46.832188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05498bae-4764-486b-8010-40f42bcd74ca admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:46.909171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05498bae-4764-486b-8010-40f42bcd74ca admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:46.915399 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13473a69-9b31-4ba8-9ab6-5ad4c05ebd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:46.916021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13473a69-9b31-4ba8-9ab6-5ad4c05ebd85 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:46.916296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13473a69-9b31-4ba8-9ab6-5ad4c05ebd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:46.916563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13473a69-9b31-4ba8-9ab6-5ad4c05ebd85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:46.917240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13473a69-9b31-4ba8-9ab6-5ad4c05ebd85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:46.919158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13473a69-9b31-4ba8-9ab6-5ad4c05ebd85 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:46.969422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1199/2386] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:15:46 2022] POST /identity/v3/projects => generated 371 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:15:46.971088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18902138-b1c7-4266-a1ad-650a57a54133 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:46.984592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18902138-b1c7-4266-a1ad-650a57a54133 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:47.072358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effb497-668b-4099-a2e6-649ef360baf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:47.073130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effb497-668b-4099-a2e6-649ef360baf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:47.073495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effb497-668b-4099-a2e6-649ef360baf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:47.073882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6effb497-668b-4099-a2e6-649ef360baf5 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:47.075205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effb497-668b-4099-a2e6-649ef360baf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:47.077358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6effb497-668b-4099-a2e6-649ef360baf5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:47.118592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1188/2387] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:15:46 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:15:47.120236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13473a69-9b31-4ba8-9ab6-5ad4c05ebd85 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:47.137636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13473a69-9b31-4ba8-9ab6-5ad4c05ebd85 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:47.148828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00061f-c059-4c24-b9d0-a8b755f7bfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:47.149467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00061f-c059-4c24-b9d0-a8b755f7bfae admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:47.149753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00061f-c059-4c24-b9d0-a8b755f7bfae admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:47.150063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00061f-c059-4c24-b9d0-a8b755f7bfae admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:47.150810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00061f-c059-4c24-b9d0-a8b755f7bfae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:47.152652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb00061f-c059-4c24-b9d0-a8b755f7bfae admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:47.176227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1200/2388] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:15:47.178413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effb497-668b-4099-a2e6-649ef360baf5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:47.206030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6effb497-668b-4099-a2e6-649ef360baf5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:47.218785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c928b4-e8a1-4934-8356-26f7011fe7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:47.220339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c928b4-e8a1-4934-8356-26f7011fe7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:47.221366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c928b4-e8a1-4934-8356-26f7011fe7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:47.221734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c928b4-e8a1-4934-8356-26f7011fe7bc admin admin] PATH_INFO: `/v3/projects/f5f7988d84444015bf5aefe496de6a18/users/aa268504d46f47aeab1b8df4a1d6d841/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:47.260099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c928b4-e8a1-4934-8356-26f7011fe7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f7988d84444015bf5aefe496de6a18, user_id=aa268504d46f47aeab1b8df4a1d6d841, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:47.265754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c928b4-e8a1-4934-8356-26f7011fe7bc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:47.291800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1189/2389] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:15:47 2022] PUT /identity/v3/projects/f5f7988d84444015bf5aefe496de6a18/users/aa268504d46f47aeab1b8df4a1d6d841/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:47.296130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00061f-c059-4c24-b9d0-a8b755f7bfae admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:47.311650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00061f-c059-4c24-b9d0-a8b755f7bfae admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:47.368759 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:47.396537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9114d-1185-4ade-bc82-2bb69d826150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:47.397279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9114d-1185-4ade-bc82-2bb69d826150 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:47.397805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9114d-1185-4ade-bc82-2bb69d826150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:47.398099 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9114d-1185-4ade-bc82-2bb69d826150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:47.398989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9114d-1185-4ade-bc82-2bb69d826150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:47.402677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9114d-1185-4ade-bc82-2bb69d826150 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:47.430913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1201/2390] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:47.437373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c928b4-e8a1-4934-8356-26f7011fe7bc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:47.451899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c928b4-e8a1-4934-8356-26f7011fe7bc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:47.458526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173785b5-d287-4503-be93-d82eeae8c5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:47.460314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173785b5-d287-4503-be93-d82eeae8c5ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:47.460816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173785b5-d287-4503-be93-d82eeae8c5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:47.461303 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173785b5-d287-4503-be93-d82eeae8c5ab admin admin] PATH_INFO: `/v3/projects/f5f7988d84444015bf5aefe496de6a18/users/aa268504d46f47aeab1b8df4a1d6d841/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:47.464305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173785b5-d287-4503-be93-d82eeae8c5ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5f7988d84444015bf5aefe496de6a18, user_id=aa268504d46f47aeab1b8df4a1d6d841, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:47.467567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173785b5-d287-4503-be93-d82eeae8c5ab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:47.472484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:47.485575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1190/2391] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:15:47 2022] PUT /identity/v3/projects/f5f7988d84444015bf5aefe496de6a18/users/aa268504d46f47aeab1b8df4a1d6d841/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:47.499550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b3e1e-a441-4f44-8110-70250c72c278 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:47.500347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b3e1e-a441-4f44-8110-70250c72c278 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:47.501406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8b3e1e-a441-4f44-8110-70250c72c278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:47.547490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7d8b3e1e-a441-4f44-8110-70250c72c278 None None] MFA Rules not processed for user `aa268504d46f47aeab1b8df4a1d6d841`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:47.549182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7d8b3e1e-a441-4f44-8110-70250c72c278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:47.696750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1202/2392] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:47.709023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173785b5-d287-4503-be93-d82eeae8c5ab admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:47.711668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-173785b5-d287-4503-be93-d82eeae8c5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:47.804728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173785b5-d287-4503-be93-d82eeae8c5ab admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:47.819268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c7b9a-5f0c-4947-b50e-90c559dbb86a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:47.820157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7b9a-5f0c-4947-b50e-90c559dbb86a service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:47.820593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7b9a-5f0c-4947-b50e-90c559dbb86a service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:47.821067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7b9a-5f0c-4947-b50e-90c559dbb86a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:47.832682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c7b9a-5f0c-4947-b50e-90c559dbb86a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:47.834669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c7b9a-5f0c-4947-b50e-90c559dbb86a service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:47.847717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1191/2393] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:15:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:15:48.562210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b3e1e-a441-4f44-8110-70250c72c278 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:48.678721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8b3e1e-a441-4f44-8110-70250c72c278 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:48.686638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:48.687792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:48.687792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:48.688097 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] PATH_INFO: `/v3/users/d13425bfe5f74ef0b8237033ec946baf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:48.689149 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=d13425bfe5f74ef0b8237033ec946baf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:48.691173 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:48.780312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd13425bfe5f74ef0b8237033ec946baf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:48.781096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd13425bfe5f74ef0b8237033ec946baf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:48.782021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd13425bfe5f74ef0b8237033ec946baf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:48.800120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd13425bfe5f74ef0b8237033ec946baf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:48.815034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:48.837369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd13425bfe5f74ef0b8237033ec946baf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:48.862427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1203/2394] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:15:48 2022] DELETE /identity/v3/users/d13425bfe5f74ef0b8237033ec946baf => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:48.863475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01aeec-b95f-4251-a86b-f456bb1bc73f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:48.863920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01aeec-b95f-4251-a86b-f456bb1bc73f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:48.864454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01aeec-b95f-4251-a86b-f456bb1bc73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:48.910074 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fa01aeec-b95f-4251-a86b-f456bb1bc73f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:48.911374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa01aeec-b95f-4251-a86b-f456bb1bc73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:49.017591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1192/2395] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:15:49.033946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:49.035989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:49.204442 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179055b2-959a-4e3b-a1b0-6bdf49f31cab admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:49.210006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5eeda-182b-4ca6-a040-b48503bc281f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:49.211017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5eeda-182b-4ca6-a040-b48503bc281f service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:49.211511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5eeda-182b-4ca6-a040-b48503bc281f service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:49.212049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5eeda-182b-4ca6-a040-b48503bc281f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:49.230995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5eeda-182b-4ca6-a040-b48503bc281f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:49.234160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5eeda-182b-4ca6-a040-b48503bc281f service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:49.263056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1204/2396] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:15:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:15:49.669744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01aeec-b95f-4251-a86b-f456bb1bc73f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:49.674166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa01aeec-b95f-4251-a86b-f456bb1bc73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:49.696419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01aeec-b95f-4251-a86b-f456bb1bc73f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:49.710657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:49.711256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:49.712535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:49.712790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] PATH_INFO: `/v3/projects/2e364e5252464d30a002dc479c82c062` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:49.714071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e364e5252464d30a002dc479c82c062)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:49.716054 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:49.787822 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e364e5252464d30a002dc479c82c062', 'request_id': 'req-b58891fc-acfb-4ca9-86db-f6f4196b9c56'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:49.795439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e364e5252464d30a002dc479c82c062', 'request_id': 'req-b58891fc-acfb-4ca9-86db-f6f4196b9c56'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:49.814735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e364e5252464d30a002dc479c82c062', 'request_id': 'req-b58891fc-acfb-4ca9-86db-f6f4196b9c56'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:49.840498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] The token cache is being invalidate because project 2e364e5252464d30a002dc479c82c062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:15:49.840843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:49.841473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:49.846369 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1193/2397] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:15:49 2022] DELETE /identity/v3/projects/2e364e5252464d30a002dc479c82c062 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:49.848833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5eeda-182b-4ca6-a040-b48503bc281f service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:49.860131 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5eeda-182b-4ca6-a040-b48503bc281f service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:49.981961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:49.983361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:49.984180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:49.984562 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] PATH_INFO: `/v3/users/aa268504d46f47aeab1b8df4a1d6d841` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:49.986677 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa268504d46f47aeab1b8df4a1d6d841)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:49.989746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:50.107546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa268504d46f47aeab1b8df4a1d6d841'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:50.108625 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa268504d46f47aeab1b8df4a1d6d841'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:50.109478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa268504d46f47aeab1b8df4a1d6d841'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:50.127025 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa268504d46f47aeab1b8df4a1d6d841'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:50.142010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75f97ff3-f208-4025-ad7f-031b27078e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa268504d46f47aeab1b8df4a1d6d841'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:50.161895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1205/2398] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:15:49 2022] DELETE /identity/v3/users/aa268504d46f47aeab1b8df4a1d6d841 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:15:50.617387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:50.619480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:50.649085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58891fc-acfb-4ca9-86db-f6f4196b9c56 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:50.694113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:50.801947 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:50.803629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:50.803987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:50.804332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] PATH_INFO: `/v3/projects/f5f7988d84444015bf5aefe496de6a18` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:50.805954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5f7988d84444015bf5aefe496de6a18)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:50.808345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:50.920482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5f7988d84444015bf5aefe496de6a18', 'request_id': 'req-9b479634-9551-4c79-b11e-4a3e0f130a3d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:50.929590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5f7988d84444015bf5aefe496de6a18', 'request_id': 'req-9b479634-9551-4c79-b11e-4a3e0f130a3d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:50.937004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5f7988d84444015bf5aefe496de6a18', 'request_id': 'req-9b479634-9551-4c79-b11e-4a3e0f130a3d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:50.974785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] The token cache is being invalidate because project f5f7988d84444015bf5aefe496de6a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:15:50.975356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:50.977425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9b479634-9551-4c79-b11e-4a3e0f130a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:15:50.979568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1194/2399] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:15:50 2022] DELETE /identity/v3/projects/f5f7988d84444015bf5aefe496de6a18 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:15:50.984609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dc2cf-4ae2-4dcf-9642-353fb4dfcc8d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:50.985066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dc2cf-4ae2-4dcf-9642-353fb4dfcc8d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:50.985149 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5dc2cf-4ae2-4dcf-9642-353fb4dfcc8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:51.000869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:51.027667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1f5dc2cf-4ae2-4dcf-9642-353fb4dfcc8d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:51.029401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1f5dc2cf-4ae2-4dcf-9642-353fb4dfcc8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:51.189202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1206/2400] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:15:51.197636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662fada-28cb-4cdc-8e2d-3e39f8d12024 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:51.198058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662fada-28cb-4cdc-8e2d-3e39f8d12024 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:51.198287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662fada-28cb-4cdc-8e2d-3e39f8d12024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:51.227936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e662fada-28cb-4cdc-8e2d-3e39f8d12024 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:51.229188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e662fada-28cb-4cdc-8e2d-3e39f8d12024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:51.260697 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1195/2401] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:51.261201 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dc2cf-4ae2-4dcf-9642-353fb4dfcc8d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:51.275750 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5dc2cf-4ae2-4dcf-9642-353fb4dfcc8d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:51.302735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273894be-7d91-4d3b-a15d-e58d74436f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:51.303686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273894be-7d91-4d3b-a15d-e58d74436f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:51.303987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273894be-7d91-4d3b-a15d-e58d74436f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:51.304364 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273894be-7d91-4d3b-a15d-e58d74436f34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:51.306016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273894be-7d91-4d3b-a15d-e58d74436f34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:15:51.306441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273894be-7d91-4d3b-a15d-e58d74436f34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:51.312773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273894be-7d91-4d3b-a15d-e58d74436f34 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:51.333495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1207/2402] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:15:51.340036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662fada-28cb-4cdc-8e2d-3e39f8d12024 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:51.359490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662fada-28cb-4cdc-8e2d-3e39f8d12024 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:51.367003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7f699-b8c6-465e-a6c4-dcdd011316ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:51.367637 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7f699-b8c6-465e-a6c4-dcdd011316ea admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:51.367912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7f699-b8c6-465e-a6c4-dcdd011316ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:51.368181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7f699-b8c6-465e-a6c4-dcdd011316ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:51.368831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7f699-b8c6-465e-a6c4-dcdd011316ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:51.370854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7f699-b8c6-465e-a6c4-dcdd011316ea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:51.419693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1196/2403] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:15:51 2022] POST /identity/v3/projects => generated 357 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:15:51.424430 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273894be-7d91-4d3b-a15d-e58d74436f34 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:51.437990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273894be-7d91-4d3b-a15d-e58d74436f34 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:51.539588 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cc747-a603-48c0-bc3a-7ab28fa7db8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:51.541421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cc747-a603-48c0-bc3a-7ab28fa7db8e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:51.541966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cc747-a603-48c0-bc3a-7ab28fa7db8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:51.543303 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cc747-a603-48c0-bc3a-7ab28fa7db8e admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:51.544175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cc747-a603-48c0-bc3a-7ab28fa7db8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:51.546903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5cc747-a603-48c0-bc3a-7ab28fa7db8e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:51.580489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1208/2404] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:15:51 2022] POST /identity/v3/users => generated 279 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:15:51.580948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7f699-b8c6-465e-a6c4-dcdd011316ea admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:51.607410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7f699-b8c6-465e-a6c4-dcdd011316ea admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:51.614092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d4985-e9c1-482c-af84-bf5c1c6e6fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:51.614957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d4985-e9c1-482c-af84-bf5c1c6e6fcb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:51.615434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d4985-e9c1-482c-af84-bf5c1c6e6fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:51.616032 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d4985-e9c1-482c-af84-bf5c1c6e6fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:51.616964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d4985-e9c1-482c-af84-bf5c1c6e6fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:51.618895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d4985-e9c1-482c-af84-bf5c1c6e6fcb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:51.647671 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cc747-a603-48c0-bc3a-7ab28fa7db8e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:51.668469 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1197/2405] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:51.677037 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cc747-a603-48c0-bc3a-7ab28fa7db8e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:51.683497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8eb6ba-8c9c-4e90-9b44-a3d54cea749f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:51.684086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8eb6ba-8c9c-4e90-9b44-a3d54cea749f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:51.684555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8eb6ba-8c9c-4e90-9b44-a3d54cea749f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:51.684924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8eb6ba-8c9c-4e90-9b44-a3d54cea749f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:51.685687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8eb6ba-8c9c-4e90-9b44-a3d54cea749f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:51.688017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8eb6ba-8c9c-4e90-9b44-a3d54cea749f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:51.707823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d4985-e9c1-482c-af84-bf5c1c6e6fcb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:51.718782 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1209/2406] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:51.723596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d4985-e9c1-482c-af84-bf5c1c6e6fcb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:51.731986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95115a5f-e17e-40a3-8558-8fcedbbcd113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:51.733193 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95115a5f-e17e-40a3-8558-8fcedbbcd113 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:51.733710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95115a5f-e17e-40a3-8558-8fcedbbcd113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:51.734207 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95115a5f-e17e-40a3-8558-8fcedbbcd113 admin admin] PATH_INFO: `/v3/projects/207d70b234ff4d47a0fef5ed25226702/users/d80446e861cc40eb89deb4acd78279f9/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:51.759309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95115a5f-e17e-40a3-8558-8fcedbbcd113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=207d70b234ff4d47a0fef5ed25226702, user_id=d80446e861cc40eb89deb4acd78279f9, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:51.763441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95115a5f-e17e-40a3-8558-8fcedbbcd113 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:51.827094 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1198/2407] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:15:51 2022] PUT /identity/v3/projects/207d70b234ff4d47a0fef5ed25226702/users/d80446e861cc40eb89deb4acd78279f9/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:51.829224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900f6bf-6172-4cce-9678-307ab6d79ca1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:51.829924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900f6bf-6172-4cce-9678-307ab6d79ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:51.830350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900f6bf-6172-4cce-9678-307ab6d79ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:51.863941 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4900f6bf-6172-4cce-9678-307ab6d79ca1 None None] MFA Rules not processed for user `d80446e861cc40eb89deb4acd78279f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:51.865995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4900f6bf-6172-4cce-9678-307ab6d79ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:51.958231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:51.976302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95115a5f-e17e-40a3-8558-8fcedbbcd113 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:51.978050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1210/2408] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:52.055834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:52.076836 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95115a5f-e17e-40a3-8558-8fcedbbcd113 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:52.088970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb9b0e-be4f-47fa-80a6-0c7b9ba2c204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:52.089693 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb9b0e-be4f-47fa-80a6-0c7b9ba2c204 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:52.090146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb9b0e-be4f-47fa-80a6-0c7b9ba2c204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:52.090520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb9b0e-be4f-47fa-80a6-0c7b9ba2c204 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:52.091274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb9b0e-be4f-47fa-80a6-0c7b9ba2c204 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:52.093926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb9b0e-be4f-47fa-80a6-0c7b9ba2c204 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:52.137385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1199/2409] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:15:51 2022] POST /identity/v3/projects => generated 355 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:15:52.137582 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900f6bf-6172-4cce-9678-307ab6d79ca1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:52.153289 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900f6bf-6172-4cce-9678-307ab6d79ca1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:52.238822 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f2602-b79c-4209-a4b9-63cb837e8ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:52.239690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f2602-b79c-4209-a4b9-63cb837e8ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:52.240715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f2602-b79c-4209-a4b9-63cb837e8ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:52.241160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f2602-b79c-4209-a4b9-63cb837e8ae8 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:52.242107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f2602-b79c-4209-a4b9-63cb837e8ae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:52.244263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f2602-b79c-4209-a4b9-63cb837e8ae8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:52.281789 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1211/2410] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:15:52 2022] POST /identity/v3/users => generated 284 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:15:52.285538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb9b0e-be4f-47fa-80a6-0c7b9ba2c204 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:52.302447 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb9b0e-be4f-47fa-80a6-0c7b9ba2c204 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:52.310179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb601a6c-4714-4049-8ae5-62900d2b8a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:52.310946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb601a6c-4714-4049-8ae5-62900d2b8a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:52.312084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb601a6c-4714-4049-8ae5-62900d2b8a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:52.312084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb601a6c-4714-4049-8ae5-62900d2b8a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:52.312533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb601a6c-4714-4049-8ae5-62900d2b8a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:52.314973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb601a6c-4714-4049-8ae5-62900d2b8a43 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:52.342519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1200/2411] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:52.343931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f2602-b79c-4209-a4b9-63cb837e8ae8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:52.360653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f2602-b79c-4209-a4b9-63cb837e8ae8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:52.368439 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc4cf8-e65c-4511-ab33-838ffdadea4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:52.369639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc4cf8-e65c-4511-ab33-838ffdadea4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:52.370502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc4cf8-e65c-4511-ab33-838ffdadea4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:52.371054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bc4cf8-e65c-4511-ab33-838ffdadea4a admin admin] PATH_INFO: `/v3/projects/4d18e014c91f499b93572cd9ada7da0d/users/0cee4f91fa7d457e86feffd352837c92/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:52.392144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc4cf8-e65c-4511-ab33-838ffdadea4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d18e014c91f499b93572cd9ada7da0d, user_id=0cee4f91fa7d457e86feffd352837c92, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:52.395610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bc4cf8-e65c-4511-ab33-838ffdadea4a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:52.429436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1212/2412] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:15:52 2022] PUT /identity/v3/projects/4d18e014c91f499b93572cd9ada7da0d/users/0cee4f91fa7d457e86feffd352837c92/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:15:52.433363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb601a6c-4714-4049-8ae5-62900d2b8a43 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:52.454662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb601a6c-4714-4049-8ae5-62900d2b8a43 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:52.576484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c2b00-2a20-4a22-920a-eba1e7723911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:52.577341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c2b00-2a20-4a22-920a-eba1e7723911 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:52.577475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c2b00-2a20-4a22-920a-eba1e7723911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:52.577743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c2b00-2a20-4a22-920a-eba1e7723911 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:52.578436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c2b00-2a20-4a22-920a-eba1e7723911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:52.580067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c2b00-2a20-4a22-920a-eba1e7723911 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:52.607267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1201/2413] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:15:52.611124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc4cf8-e65c-4511-ab33-838ffdadea4a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:52.627558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bc4cf8-e65c-4511-ab33-838ffdadea4a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:52.640551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845d946-d583-4788-8bb1-128ee8b1c091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:52.641392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845d946-d583-4788-8bb1-128ee8b1c091 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:52.641649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845d946-d583-4788-8bb1-128ee8b1c091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:52.641952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2845d946-d583-4788-8bb1-128ee8b1c091 admin admin] PATH_INFO: `/v3/projects/4d18e014c91f499b93572cd9ada7da0d/users/0cee4f91fa7d457e86feffd352837c92/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:52.645538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845d946-d583-4788-8bb1-128ee8b1c091 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d18e014c91f499b93572cd9ada7da0d, user_id=0cee4f91fa7d457e86feffd352837c92, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:52.654878 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2845d946-d583-4788-8bb1-128ee8b1c091 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:52.678491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1213/2414] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:15:52 2022] PUT /identity/v3/projects/4d18e014c91f499b93572cd9ada7da0d/users/0cee4f91fa7d457e86feffd352837c92/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:15:52.689350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70622f3d-d398-4e75-8572-2256d2784147 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:52.689917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70622f3d-d398-4e75-8572-2256d2784147 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:52.690379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70622f3d-d398-4e75-8572-2256d2784147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:52.751206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-70622f3d-d398-4e75-8572-2256d2784147 None None] MFA Rules not processed for user `0cee4f91fa7d457e86feffd352837c92`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:52.753033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-70622f3d-d398-4e75-8572-2256d2784147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:52.866166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1202/2415] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:52 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:52.877180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845d946-d583-4788-8bb1-128ee8b1c091 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:52.879140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2845d946-d583-4788-8bb1-128ee8b1c091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:52.991170 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2845d946-d583-4788-8bb1-128ee8b1c091 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:52.999235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5fc6f-3d54-42c4-8ece-daa0f7374272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:53.000231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5fc6f-3d54-42c4-8ece-daa0f7374272 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:53.000678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5fc6f-3d54-42c4-8ece-daa0f7374272 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:53.001115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5fc6f-3d54-42c4-8ece-daa0f7374272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:53.019487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5fc6f-3d54-42c4-8ece-daa0f7374272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:53.022652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5fc6f-3d54-42c4-8ece-daa0f7374272 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:53.033936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1214/2416] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:15:52 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:15:53.140339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70622f3d-d398-4e75-8572-2256d2784147 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:53.156439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70622f3d-d398-4e75-8572-2256d2784147 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:53.167211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d860511-b1d2-47bd-adb7-6ed2e01acada service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:53.167854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d860511-b1d2-47bd-adb7-6ed2e01acada service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:53.168116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d860511-b1d2-47bd-adb7-6ed2e01acada service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:53.168405 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d860511-b1d2-47bd-adb7-6ed2e01acada service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:53.228863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:53.256712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d860511-b1d2-47bd-adb7-6ed2e01acada service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:53.258654 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d860511-b1d2-47bd-adb7-6ed2e01acada service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:53.276006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1203/2417] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:15:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:15:53.751508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e6aec-9a7e-4081-94a2-cd6205f6c650 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:53.751962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e6aec-9a7e-4081-94a2-cd6205f6c650 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:53.752045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e6aec-9a7e-4081-94a2-cd6205f6c650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:53.796687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-397e6aec-9a7e-4081-94a2-cd6205f6c650 None None] MFA Rules not processed for user `b329612022194c9dacf15ec93814fcfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:53.797982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-397e6aec-9a7e-4081-94a2-cd6205f6c650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:53.867721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:15:53.912738 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1215/2418] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:15:53.930544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d860511-b1d2-47bd-adb7-6ed2e01acada service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:15:53.931910 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2d860511-b1d2-47bd-adb7-6ed2e01acada service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:54.074422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d860511-b1d2-47bd-adb7-6ed2e01acada service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:15:54.085223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c549a71-6103-4ce1-bdfd-79b9598aef70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:15:54.086238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c549a71-6103-4ce1-bdfd-79b9598aef70 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:54.086873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c549a71-6103-4ce1-bdfd-79b9598aef70 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:54.087701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c549a71-6103-4ce1-bdfd-79b9598aef70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:54.091290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c549a71-6103-4ce1-bdfd-79b9598aef70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:54.195557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c549a71-6103-4ce1-bdfd-79b9598aef70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:15:54.197299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c549a71-6103-4ce1-bdfd-79b9598aef70 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:15:54.208128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1204/2419] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:15:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:15:55.594839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1b0f0-72d8-4522-8106-af8601eaa232 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:15:55.596080 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1b0f0-72d8-4522-8106-af8601eaa232 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:15:55.596641 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1b0f0-72d8-4522-8106-af8601eaa232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:15:55.620595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-79f1b0f0-72d8-4522-8106-af8601eaa232 None None] MFA Rules not processed for user `f7ed8b3dc88c4fadbb0839fd848397d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:15:55.622041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-79f1b0f0-72d8-4522-8106-af8601eaa232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:15:55.639795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1216/2420] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:16:04.203668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=65 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:05.612723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=13 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:10.855280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c549a71-6103-4ce1-bdfd-79b9598aef70 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:10.972784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c549a71-6103-4ce1-bdfd-79b9598aef70 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:10.978726 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:10.979371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:10.979647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:10.979923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] PATH_INFO: `/v3/users/d80446e861cc40eb89deb4acd78279f9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:10.981147 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d80446e861cc40eb89deb4acd78279f9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:10.983104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:11.043337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd80446e861cc40eb89deb4acd78279f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.051420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd80446e861cc40eb89deb4acd78279f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.052089 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd80446e861cc40eb89deb4acd78279f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.052702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd80446e861cc40eb89deb4acd78279f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.061266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd80446e861cc40eb89deb4acd78279f9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.085914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1205/2421] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:16:10 2022] DELETE /identity/v3/users/d80446e861cc40eb89deb4acd78279f9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:11.090910 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd9917-5c62-45a7-b3e0-061cc783739f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:11.091434 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd9917-5c62-45a7-b3e0-061cc783739f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:11.092012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd9917-5c62-45a7-b3e0-061cc783739f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:11.149345 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-11dd9917-5c62-45a7-b3e0-061cc783739f None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:11.150508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11dd9917-5c62-45a7-b3e0-061cc783739f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:11.220389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1217/2422] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:16:11.231592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:11.233044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:11.279314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:11.314386 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ae959-5912-4fbc-85d5-dc0bc95f75ed admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:11.318057 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814f584-1170-4596-a9f8-ffdb321c93f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:11.318659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814f584-1170-4596-a9f8-ffdb321c93f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:11.318936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814f584-1170-4596-a9f8-ffdb321c93f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:11.319205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814f584-1170-4596-a9f8-ffdb321c93f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:11.329550 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814f584-1170-4596-a9f8-ffdb321c93f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:11.331027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b814f584-1170-4596-a9f8-ffdb321c93f6 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:11.342129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1206/2423] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:16:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:16:11.643425 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd9917-5c62-45a7-b3e0-061cc783739f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:11.644944 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11dd9917-5c62-45a7-b3e0-061cc783739f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:11.667658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd9917-5c62-45a7-b3e0-061cc783739f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:11.679567 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:11.679971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:11.680265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:11.680654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] PATH_INFO: `/v3/projects/207d70b234ff4d47a0fef5ed25226702` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:11.681869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=207d70b234ff4d47a0fef5ed25226702)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:11.683792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:11.737358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '207d70b234ff4d47a0fef5ed25226702', 'request_id': 'req-9d55e0bc-da91-4b26-8e69-37f575224c0e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.745716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '207d70b234ff4d47a0fef5ed25226702', 'request_id': 'req-9d55e0bc-da91-4b26-8e69-37f575224c0e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.752127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '207d70b234ff4d47a0fef5ed25226702', 'request_id': 'req-9d55e0bc-da91-4b26-8e69-37f575224c0e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.764433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] The token cache is being invalidate because project 207d70b234ff4d47a0fef5ed25226702 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:16:11.764725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.765398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.766945 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1218/2424] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:16:11 2022] DELETE /identity/v3/projects/207d70b234ff4d47a0fef5ed25226702 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:11.771212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814f584-1170-4596-a9f8-ffdb321c93f6 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:11.782983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814f584-1170-4596-a9f8-ffdb321c93f6 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:11.855514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:11.856600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:11.857138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:11.857647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] PATH_INFO: `/v3/users/0cee4f91fa7d457e86feffd352837c92` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:11.859736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cee4f91fa7d457e86feffd352837c92)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:11.863349 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:11.926082 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cee4f91fa7d457e86feffd352837c92'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.934744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cee4f91fa7d457e86feffd352837c92'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.935385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cee4f91fa7d457e86feffd352837c92'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.936041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cee4f91fa7d457e86feffd352837c92'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.945228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-76064b95-da23-4d0a-9a1d-348417ec360d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cee4f91fa7d457e86feffd352837c92'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:11.963911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1207/2425] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:16:11 2022] DELETE /identity/v3/users/0cee4f91fa7d457e86feffd352837c92 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:12.272348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:12.275436 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:12.291087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55e0bc-da91-4b26-8e69-37f575224c0e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:12.359865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:12.367942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:12.368807 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:12.369554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:12.369966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] PATH_INFO: `/v3/projects/4d18e014c91f499b93572cd9ada7da0d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:12.371342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d18e014c91f499b93572cd9ada7da0d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:12.373412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:12.431810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d18e014c91f499b93572cd9ada7da0d', 'request_id': 'req-0cb37d89-a567-46d0-866d-5df833cb4d49'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:12.439539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d18e014c91f499b93572cd9ada7da0d', 'request_id': 'req-0cb37d89-a567-46d0-866d-5df833cb4d49'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:12.449820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d18e014c91f499b93572cd9ada7da0d', 'request_id': 'req-0cb37d89-a567-46d0-866d-5df833cb4d49'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:12.463592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] The token cache is being invalidate because project 4d18e014c91f499b93572cd9ada7da0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:16:12.463888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:12.464575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0cb37d89-a567-46d0-866d-5df833cb4d49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:12.466114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1219/2426] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:16:12 2022] DELETE /identity/v3/projects/4d18e014c91f499b93572cd9ada7da0d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:12.471152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a6e7-18b5-4418-9b3f-d8ae5e10d38a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:12.471740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a6e7-18b5-4418-9b3f-d8ae5e10d38a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:12.472123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a6e7-18b5-4418-9b3f-d8ae5e10d38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:12.534537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8966a6e7-18b5-4418-9b3f-d8ae5e10d38a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:12.535686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8966a6e7-18b5-4418-9b3f-d8ae5e10d38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:12.549635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:12.602554 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1208/2427] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:16:12.610541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7dc29-4392-4aef-b96f-9102725b9ce4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:12.610855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7dc29-4392-4aef-b96f-9102725b9ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:12.611128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7dc29-4392-4aef-b96f-9102725b9ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:12.633300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-dfa7dc29-4392-4aef-b96f-9102725b9ce4 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:12.634648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dfa7dc29-4392-4aef-b96f-9102725b9ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:12.643225 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1220/2428] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:16:12.647565 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966a6e7-18b5-4418-9b3f-d8ae5e10d38a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:12.659010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966a6e7-18b5-4418-9b3f-d8ae5e10d38a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:12.664200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ff18a-e541-4eb8-9fc0-35ac4418ae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:12.664804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ff18a-e541-4eb8-9fc0-35ac4418ae54 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:12.665079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ff18a-e541-4eb8-9fc0-35ac4418ae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:12.665348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ff18a-e541-4eb8-9fc0-35ac4418ae54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:12.666036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ff18a-e541-4eb8-9fc0-35ac4418ae54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:16:12.666338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ff18a-e541-4eb8-9fc0-35ac4418ae54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:12.667959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ff18a-e541-4eb8-9fc0-35ac4418ae54 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:12.680889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1209/2429] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:16:12.686423 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7dc29-4392-4aef-b96f-9102725b9ce4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:12.704192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7dc29-4392-4aef-b96f-9102725b9ce4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:12.712398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd7407-45fb-4ff9-a0a3-0bca4e319f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:12.713229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd7407-45fb-4ff9-a0a3-0bca4e319f13 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:12.713674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd7407-45fb-4ff9-a0a3-0bca4e319f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:12.714218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd7407-45fb-4ff9-a0a3-0bca4e319f13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:12.715260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd7407-45fb-4ff9-a0a3-0bca4e319f13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:12.719127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd7407-45fb-4ff9-a0a3-0bca4e319f13 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:12.763117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1221/2430] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:16:12 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:16:12.767284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ff18a-e541-4eb8-9fc0-35ac4418ae54 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:12.777828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ff18a-e541-4eb8-9fc0-35ac4418ae54 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:12.841946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc686571-112d-4984-b2b7-1a09f6d11918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:12.842473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686571-112d-4984-b2b7-1a09f6d11918 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:12.842877 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686571-112d-4984-b2b7-1a09f6d11918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:12.843023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc686571-112d-4984-b2b7-1a09f6d11918 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:12.843669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc686571-112d-4984-b2b7-1a09f6d11918 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:12.845609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc686571-112d-4984-b2b7-1a09f6d11918 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:12.870475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1210/2431] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:16:12 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:16:12.874621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd7407-45fb-4ff9-a0a3-0bca4e319f13 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:12.885471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd7407-45fb-4ff9-a0a3-0bca4e319f13 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:12.891325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cece48b-59d8-42c2-8088-b28850ca38b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:12.891942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cece48b-59d8-42c2-8088-b28850ca38b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:12.892218 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cece48b-59d8-42c2-8088-b28850ca38b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:12.892488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cece48b-59d8-42c2-8088-b28850ca38b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:12.893132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cece48b-59d8-42c2-8088-b28850ca38b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:12.894806 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cece48b-59d8-42c2-8088-b28850ca38b6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:12.905471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1222/2432] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:16:12.909885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc686571-112d-4984-b2b7-1a09f6d11918 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:12.922044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc686571-112d-4984-b2b7-1a09f6d11918 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:12.927150 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfc25b-93d8-42f9-b590-e55fcbc29cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:12.927784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfc25b-93d8-42f9-b590-e55fcbc29cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:12.928205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfc25b-93d8-42f9-b590-e55fcbc29cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:12.928333 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfc25b-93d8-42f9-b590-e55fcbc29cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:12.929723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfc25b-93d8-42f9-b590-e55fcbc29cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:12.932033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfc25b-93d8-42f9-b590-e55fcbc29cf6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:12.944262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1211/2433] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:16:12.949037 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cece48b-59d8-42c2-8088-b28850ca38b6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:12.959394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cece48b-59d8-42c2-8088-b28850ca38b6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:12.965392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221db2b-add2-4b1a-8952-3b7148e824c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:12.966104 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221db2b-add2-4b1a-8952-3b7148e824c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:12.966379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221db2b-add2-4b1a-8952-3b7148e824c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:12.966651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b221db2b-add2-4b1a-8952-3b7148e824c9 admin admin] PATH_INFO: `/v3/projects/d698af572d1c47189f920d8d8bb91431/users/76846c6b94f74e29ae1df8b3894203e3/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:12.984239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221db2b-add2-4b1a-8952-3b7148e824c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d698af572d1c47189f920d8d8bb91431, user_id=76846c6b94f74e29ae1df8b3894203e3, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:12.986947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b221db2b-add2-4b1a-8952-3b7148e824c9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:13.001715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1223/2434] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:16:12 2022] PUT /identity/v3/projects/d698af572d1c47189f920d8d8bb91431/users/76846c6b94f74e29ae1df8b3894203e3/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:13.014246 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe983ec8-17d2-43fe-9042-3ef5d96f45e0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:13.014529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe983ec8-17d2-43fe-9042-3ef5d96f45e0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:13.014824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe983ec8-17d2-43fe-9042-3ef5d96f45e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:13.045960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fe983ec8-17d2-43fe-9042-3ef5d96f45e0 None None] MFA Rules not processed for user `76846c6b94f74e29ae1df8b3894203e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:13.047187 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe983ec8-17d2-43fe-9042-3ef5d96f45e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:13.118035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1212/2435] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:16:13.122983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221db2b-add2-4b1a-8952-3b7148e824c9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:13.197081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b221db2b-add2-4b1a-8952-3b7148e824c9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:13.203072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1ccad-bb3d-4e97-b51b-6c9cdcd67665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:13.203716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ccad-bb3d-4e97-b51b-6c9cdcd67665 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:13.204182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ccad-bb3d-4e97-b51b-6c9cdcd67665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:13.204464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ccad-bb3d-4e97-b51b-6c9cdcd67665 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:13.205114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1ccad-bb3d-4e97-b51b-6c9cdcd67665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:13.207090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1ccad-bb3d-4e97-b51b-6c9cdcd67665 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:13.238408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1224/2436] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:16:13 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:16:13.243809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe983ec8-17d2-43fe-9042-3ef5d96f45e0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:13.261959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe983ec8-17d2-43fe-9042-3ef5d96f45e0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:13.274985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:13.328799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86ab12-39c5-4da9-bcf7-2a9e5aa3e1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:13.329416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86ab12-39c5-4da9-bcf7-2a9e5aa3e1ed admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:13.329696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86ab12-39c5-4da9-bcf7-2a9e5aa3e1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:13.330004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86ab12-39c5-4da9-bcf7-2a9e5aa3e1ed admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:13.330636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86ab12-39c5-4da9-bcf7-2a9e5aa3e1ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:13.332578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86ab12-39c5-4da9-bcf7-2a9e5aa3e1ed admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:13.362853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1213/2437] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:16:13 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:16:13.364876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1ccad-bb3d-4e97-b51b-6c9cdcd67665 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:13.377819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1ccad-bb3d-4e97-b51b-6c9cdcd67665 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:13.383458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e2fa-bcc0-4834-8b01-bd5210db8755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:13.384129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e2fa-bcc0-4834-8b01-bd5210db8755 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:13.384410 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e2fa-bcc0-4834-8b01-bd5210db8755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:13.384679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e2fa-bcc0-4834-8b01-bd5210db8755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:13.385412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98e2fa-bcc0-4834-8b01-bd5210db8755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:13.386974 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98e2fa-bcc0-4834-8b01-bd5210db8755 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:13.389817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:13.401708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1225/2438] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:16:13.405749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86ab12-39c5-4da9-bcf7-2a9e5aa3e1ed admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:13.416803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86ab12-39c5-4da9-bcf7-2a9e5aa3e1ed admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:13.421661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8863cd-6f2f-4cac-90d7-e4ecff46a6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:13.422349 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8863cd-6f2f-4cac-90d7-e4ecff46a6ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:13.422632 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8863cd-6f2f-4cac-90d7-e4ecff46a6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:13.422906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8863cd-6f2f-4cac-90d7-e4ecff46a6ed admin admin] PATH_INFO: `/v3/projects/257f2d5e54534e179030eff4ff90507d/users/36ea04a19eff4b6bb31e52ee8c089510/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:13.439321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8863cd-6f2f-4cac-90d7-e4ecff46a6ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=257f2d5e54534e179030eff4ff90507d, user_id=36ea04a19eff4b6bb31e52ee8c089510, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:13.442064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8863cd-6f2f-4cac-90d7-e4ecff46a6ed admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:13.456927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1214/2439] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:16:13 2022] PUT /identity/v3/projects/257f2d5e54534e179030eff4ff90507d/users/36ea04a19eff4b6bb31e52ee8c089510/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:13.461716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e2fa-bcc0-4834-8b01-bd5210db8755 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:13.472051 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e2fa-bcc0-4834-8b01-bd5210db8755 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:13.543095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4f29a-10b3-4007-84b7-10b9098cee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:13.543713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4f29a-10b3-4007-84b7-10b9098cee5e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:13.543997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4f29a-10b3-4007-84b7-10b9098cee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:13.544271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4f29a-10b3-4007-84b7-10b9098cee5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:13.544924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4f29a-10b3-4007-84b7-10b9098cee5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:13.546584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4f29a-10b3-4007-84b7-10b9098cee5e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:13.558594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1226/2440] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:16:13.563813 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8863cd-6f2f-4cac-90d7-e4ecff46a6ed admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:13.575091 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8863cd-6f2f-4cac-90d7-e4ecff46a6ed admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:13.580592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647481f-2ad2-4edb-a62c-79f7729aa556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:13.581388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647481f-2ad2-4edb-a62c-79f7729aa556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:13.581662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647481f-2ad2-4edb-a62c-79f7729aa556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:13.581965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5647481f-2ad2-4edb-a62c-79f7729aa556 admin admin] PATH_INFO: `/v3/projects/257f2d5e54534e179030eff4ff90507d/users/36ea04a19eff4b6bb31e52ee8c089510/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:13.583954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647481f-2ad2-4edb-a62c-79f7729aa556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=257f2d5e54534e179030eff4ff90507d, user_id=36ea04a19eff4b6bb31e52ee8c089510, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:13.587613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5647481f-2ad2-4edb-a62c-79f7729aa556 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:13.598837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1215/2441] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:16:13 2022] PUT /identity/v3/projects/257f2d5e54534e179030eff4ff90507d/users/36ea04a19eff4b6bb31e52ee8c089510/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:13.611337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6b40d-5854-44f6-9ece-520dc781d4cf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:13.611619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6b40d-5854-44f6-9ece-520dc781d4cf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:13.611912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e6b40d-5854-44f6-9ece-520dc781d4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:13.642959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d8e6b40d-5854-44f6-9ece-520dc781d4cf None None] MFA Rules not processed for user `36ea04a19eff4b6bb31e52ee8c089510`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:13.644187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d8e6b40d-5854-44f6-9ece-520dc781d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:13.726678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1227/2442] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:16:13.739839 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647481f-2ad2-4edb-a62c-79f7729aa556 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:13.741929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5647481f-2ad2-4edb-a62c-79f7729aa556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:13.819279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5647481f-2ad2-4edb-a62c-79f7729aa556 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:13.825322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1d4b-ba3a-494b-9946-59b2a2de108e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:13.826005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1d4b-ba3a-494b-9946-59b2a2de108e service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:13.826284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1d4b-ba3a-494b-9946-59b2a2de108e service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:13.826556 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb1d4b-ba3a-494b-9946-59b2a2de108e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:13.892649 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb1d4b-ba3a-494b-9946-59b2a2de108e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:13.894040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb1d4b-ba3a-494b-9946-59b2a2de108e service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:13.904217 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1216/2443] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:16:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:16:21.354437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6b40d-5854-44f6-9ece-520dc781d4cf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:21.355849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d8e6b40d-5854-44f6-9ece-520dc781d4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:21.443959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:21.449890 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e6b40d-5854-44f6-9ece-520dc781d4cf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:21.454909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a80de-9863-447a-b199-1c895d1b5a26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:21.455527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a80de-9863-447a-b199-1c895d1b5a26 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:21.455811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a80de-9863-447a-b199-1c895d1b5a26 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:21.456111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a80de-9863-447a-b199-1c895d1b5a26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:21.458631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-af2a80de-9863-447a-b199-1c895d1b5a26 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:21.471829 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a80de-9863-447a-b199-1c895d1b5a26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:21.473468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a80de-9863-447a-b199-1c895d1b5a26 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:21.485805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1228/2444] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:16:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:16:21.717686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1d4b-ba3a-494b-9946-59b2a2de108e service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:21.719143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-57eb1d4b-ba3a-494b-9946-59b2a2de108e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:21.731503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb1d4b-ba3a-494b-9946-59b2a2de108e service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:21.735238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc519c6c-ccc5-4067-8afa-c414de813ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:21.735851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc519c6c-ccc5-4067-8afa-c414de813ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:21.736129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc519c6c-ccc5-4067-8afa-c414de813ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:21.736494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc519c6c-ccc5-4067-8afa-c414de813ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:21.748278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc519c6c-ccc5-4067-8afa-c414de813ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:21.748555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc519c6c-ccc5-4067-8afa-c414de813ca2 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:21.759537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1217/2445] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:16:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:16:31.480812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:31.754181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:54.309065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a80de-9863-447a-b199-1c895d1b5a26 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:54.395883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a80de-9863-447a-b199-1c895d1b5a26 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:54.403801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:54.404352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:54.405509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:54.405893 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] PATH_INFO: `/v3/users/76846c6b94f74e29ae1df8b3894203e3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:54.416255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76846c6b94f74e29ae1df8b3894203e3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:54.418428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:54.490148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76846c6b94f74e29ae1df8b3894203e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:54.491025 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76846c6b94f74e29ae1df8b3894203e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:54.491740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76846c6b94f74e29ae1df8b3894203e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:54.502642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76846c6b94f74e29ae1df8b3894203e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:54.520779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76846c6b94f74e29ae1df8b3894203e3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:54.535695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1229/2446] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:16:54 2022] DELETE /identity/v3/users/76846c6b94f74e29ae1df8b3894203e3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:54.544594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18d61c-5a10-4ed0-b7c7-8d676102832d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:54.546339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18d61c-5a10-4ed0-b7c7-8d676102832d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:54.546871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f18d61c-5a10-4ed0-b7c7-8d676102832d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:54.582409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0f18d61c-5a10-4ed0-b7c7-8d676102832d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:54.584682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0f18d61c-5a10-4ed0-b7c7-8d676102832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:54.678155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1218/2447] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:16:54.690480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:54.692731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:54.785135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c2503-13cf-4485-98b3-1809336e23b6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:54.788892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd7c9e-5fa0-4c8d-abe8-0a9992e76375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:54.789503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd7c9e-5fa0-4c8d-abe8-0a9992e76375 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:54.789805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd7c9e-5fa0-4c8d-abe8-0a9992e76375 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:54.790084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd7c9e-5fa0-4c8d-abe8-0a9992e76375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:54.800689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd7c9e-5fa0-4c8d-abe8-0a9992e76375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:54.802259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd7c9e-5fa0-4c8d-abe8-0a9992e76375 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:54.813917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1230/2448] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:16:55.198812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18d61c-5a10-4ed0-b7c7-8d676102832d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:55.200363 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0f18d61c-5a10-4ed0-b7c7-8d676102832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:55.214102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f18d61c-5a10-4ed0-b7c7-8d676102832d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:55.219289 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:55.219969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:55.221184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:55.221466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] PATH_INFO: `/v3/projects/d698af572d1c47189f920d8d8bb91431` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:55.223080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d698af572d1c47189f920d8d8bb91431)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:55.225111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:55.292984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd698af572d1c47189f920d8d8bb91431', 'request_id': 'req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:55.299634 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd698af572d1c47189f920d8d8bb91431', 'request_id': 'req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:55.307338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd698af572d1c47189f920d8d8bb91431', 'request_id': 'req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:55.323530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] The token cache is being invalidate because project d698af572d1c47189f920d8d8bb91431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:16:55.323875 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:55.324727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:55.326392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1219/2449] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:16:55 2022] DELETE /identity/v3/projects/d698af572d1c47189f920d8d8bb91431 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:16:55.330599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd7c9e-5fa0-4c8d-abe8-0a9992e76375 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:55.337189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:55.342460 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd7c9e-5fa0-4c8d-abe8-0a9992e76375 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:55.417730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:55.418811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:55.419315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:55.419819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] PATH_INFO: `/v3/users/36ea04a19eff4b6bb31e52ee8c089510` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:55.421745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ea04a19eff4b6bb31e52ee8c089510)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:55.425650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:55.478833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ea04a19eff4b6bb31e52ee8c089510'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:55.479466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ea04a19eff4b6bb31e52ee8c089510'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:55.480247 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ea04a19eff4b6bb31e52ee8c089510'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:55.496861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ea04a19eff4b6bb31e52ee8c089510'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:55.511359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6604602c-6c53-46c4-b6f6-a0cf808f3e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ea04a19eff4b6bb31e52ee8c089510'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:55.526857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1231/2450] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:16:55 2022] DELETE /identity/v3/users/36ea04a19eff4b6bb31e52ee8c089510 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:55.860492 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:55.863122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:55.885144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b7ef9-d6f8-4259-9e9e-0c9fb1b51045 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:55.959175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:55.959920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:55.960217 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:55.960495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] PATH_INFO: `/v3/projects/257f2d5e54534e179030eff4ff90507d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:55.961721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=257f2d5e54534e179030eff4ff90507d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:55.964209 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:56.018026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:56.043942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '257f2d5e54534e179030eff4ff90507d', 'request_id': 'req-9e944968-a42d-43ab-b651-86fbaf48b9cd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:56.050510 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '257f2d5e54534e179030eff4ff90507d', 'request_id': 'req-9e944968-a42d-43ab-b651-86fbaf48b9cd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:56.056392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '257f2d5e54534e179030eff4ff90507d', 'request_id': 'req-9e944968-a42d-43ab-b651-86fbaf48b9cd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:56.070160 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] The token cache is being invalidate because project 257f2d5e54534e179030eff4ff90507d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:16:56.070471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:56.071133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9e944968-a42d-43ab-b651-86fbaf48b9cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:56.073632 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1220/2451] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:16:55 2022] DELETE /identity/v3/projects/257f2d5e54534e179030eff4ff90507d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:56.078731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcd37b-2348-4bed-8db9-8f93fa6b5391 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.079133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcd37b-2348-4bed-8db9-8f93fa6b5391 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.079273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcd37b-2348-4bed-8db9-8f93fa6b5391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.100392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fefcd37b-2348-4bed-8db9-8f93fa6b5391 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:56.102365 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fefcd37b-2348-4bed-8db9-8f93fa6b5391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:56.181779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1232/2452] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:16:56.189997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877a4e7-91ad-4909-9e5d-a66ef340982e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.190377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877a4e7-91ad-4909-9e5d-a66ef340982e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.190722 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1877a4e7-91ad-4909-9e5d-a66ef340982e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.215561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1877a4e7-91ad-4909-9e5d-a66ef340982e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:56.216816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1877a4e7-91ad-4909-9e5d-a66ef340982e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:56.225375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1221/2453] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:16:56.230158 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcd37b-2348-4bed-8db9-8f93fa6b5391 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:56.240537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefcd37b-2348-4bed-8db9-8f93fa6b5391 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:56.246117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cf1b1-d517-44ae-986b-97566b74dbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:56.246735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cf1b1-d517-44ae-986b-97566b74dbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.247203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cf1b1-d517-44ae-986b-97566b74dbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.247459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940cf1b1-d517-44ae-986b-97566b74dbd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.248108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940cf1b1-d517-44ae-986b-97566b74dbd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:16:56.248408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940cf1b1-d517-44ae-986b-97566b74dbd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:56.250048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940cf1b1-d517-44ae-986b-97566b74dbd7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:56.263508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1233/2454] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:16:56.268406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877a4e7-91ad-4909-9e5d-a66ef340982e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:56.278516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1877a4e7-91ad-4909-9e5d-a66ef340982e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:56.283593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6830ac3-19aa-4926-8070-7017b7e7fff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:56.284192 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6830ac3-19aa-4926-8070-7017b7e7fff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.284402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6830ac3-19aa-4926-8070-7017b7e7fff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.284785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6830ac3-19aa-4926-8070-7017b7e7fff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.285354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6830ac3-19aa-4926-8070-7017b7e7fff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:56.287382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6830ac3-19aa-4926-8070-7017b7e7fff8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:56.321143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1222/2455] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:16:56 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:16:56.326070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cf1b1-d517-44ae-986b-97566b74dbd7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:56.336629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940cf1b1-d517-44ae-986b-97566b74dbd7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:56.342711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:56.428655 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb56bb-7081-431c-9f36-65573cb54b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:56.429269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb56bb-7081-431c-9f36-65573cb54b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.429548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb56bb-7081-431c-9f36-65573cb54b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.429855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb56bb-7081-431c-9f36-65573cb54b7e admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.430507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb56bb-7081-431c-9f36-65573cb54b7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:56.432722 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb56bb-7081-431c-9f36-65573cb54b7e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:56.456214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1234/2456] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:16:56 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:16:56.461680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6830ac3-19aa-4926-8070-7017b7e7fff8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:56.472792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6830ac3-19aa-4926-8070-7017b7e7fff8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:56.477425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26aa132-04e0-4870-b221-442a412cf67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:56.478692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aa132-04e0-4870-b221-442a412cf67f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.479141 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aa132-04e0-4870-b221-442a412cf67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.479420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26aa132-04e0-4870-b221-442a412cf67f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.480064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26aa132-04e0-4870-b221-442a412cf67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:56.481659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26aa132-04e0-4870-b221-442a412cf67f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:56.492874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1223/2457] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:16:56.497037 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb56bb-7081-431c-9f36-65573cb54b7e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:56.508748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb56bb-7081-431c-9f36-65573cb54b7e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:56.514003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5915616-3153-4aa8-9cda-8488a51221a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:56.514609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5915616-3153-4aa8-9cda-8488a51221a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.514885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5915616-3153-4aa8-9cda-8488a51221a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.515157 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5915616-3153-4aa8-9cda-8488a51221a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.515799 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5915616-3153-4aa8-9cda-8488a51221a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:56.517427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5915616-3153-4aa8-9cda-8488a51221a5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:56.529090 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1235/2458] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:16:56.533231 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26aa132-04e0-4870-b221-442a412cf67f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:56.544626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26aa132-04e0-4870-b221-442a412cf67f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:56.549562 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35694ece-45c3-4ae0-9d60-3877e8ae5a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:56.550248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35694ece-45c3-4ae0-9d60-3877e8ae5a27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.550529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35694ece-45c3-4ae0-9d60-3877e8ae5a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.550803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35694ece-45c3-4ae0-9d60-3877e8ae5a27 admin admin] PATH_INFO: `/v3/projects/93567d70d1984d0989c855fbfb356095/users/1e21709066d44a929064a7406dde2fc4/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.566638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35694ece-45c3-4ae0-9d60-3877e8ae5a27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93567d70d1984d0989c855fbfb356095, user_id=1e21709066d44a929064a7406dde2fc4, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:56.569702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35694ece-45c3-4ae0-9d60-3877e8ae5a27 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:56.583779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1224/2459] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:16:56 2022] PUT /identity/v3/projects/93567d70d1984d0989c855fbfb356095/users/1e21709066d44a929064a7406dde2fc4/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:56.594921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92438590-78b1-4f55-adc9-e8b94628008b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.595289 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92438590-78b1-4f55-adc9-e8b94628008b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.595661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92438590-78b1-4f55-adc9-e8b94628008b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.625156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-92438590-78b1-4f55-adc9-e8b94628008b None None] MFA Rules not processed for user `1e21709066d44a929064a7406dde2fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:56.626432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-92438590-78b1-4f55-adc9-e8b94628008b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:56.706590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1236/2460] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:16:56.711205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35694ece-45c3-4ae0-9d60-3877e8ae5a27 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:56.782392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35694ece-45c3-4ae0-9d60-3877e8ae5a27 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:56.787022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4441-5899-4def-86cf-6ff9cc36b548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:56.787631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4441-5899-4def-86cf-6ff9cc36b548 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.787912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4441-5899-4def-86cf-6ff9cc36b548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.788190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b4441-5899-4def-86cf-6ff9cc36b548 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.788835 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b4441-5899-4def-86cf-6ff9cc36b548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:56.791174 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b4441-5899-4def-86cf-6ff9cc36b548 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:56.818545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1225/2461] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:16:56 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:16:56.823743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92438590-78b1-4f55-adc9-e8b94628008b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:56.834680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92438590-78b1-4f55-adc9-e8b94628008b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:56.900653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa05ab-8178-47a8-ae8f-7d6cd80919d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:56.901352 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa05ab-8178-47a8-ae8f-7d6cd80919d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.901636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa05ab-8178-47a8-ae8f-7d6cd80919d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.901940 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa05ab-8178-47a8-ae8f-7d6cd80919d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.902584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa05ab-8178-47a8-ae8f-7d6cd80919d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:56.904534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa05ab-8178-47a8-ae8f-7d6cd80919d4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:56.925740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1237/2462] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:16:56 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:16:56.929718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4441-5899-4def-86cf-6ff9cc36b548 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:56.942784 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b4441-5899-4def-86cf-6ff9cc36b548 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:56.949522 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dbb93-bb65-44d5-90d4-cad02f21b557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:56.950175 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dbb93-bb65-44d5-90d4-cad02f21b557 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.950453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dbb93-bb65-44d5-90d4-cad02f21b557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.950726 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320dbb93-bb65-44d5-90d4-cad02f21b557 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.951380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320dbb93-bb65-44d5-90d4-cad02f21b557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:56.953003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320dbb93-bb65-44d5-90d4-cad02f21b557 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:56.965112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1226/2463] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:16:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:16:56.969306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa05ab-8178-47a8-ae8f-7d6cd80919d4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:56.980258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa05ab-8178-47a8-ae8f-7d6cd80919d4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:56.984952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef5e67-a520-42e7-8c0c-2502fbe3cec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:56.985600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef5e67-a520-42e7-8c0c-2502fbe3cec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:56.985913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef5e67-a520-42e7-8c0c-2502fbe3cec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:56.986190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ef5e67-a520-42e7-8c0c-2502fbe3cec8 admin admin] PATH_INFO: `/v3/projects/a329106baf0d4015b75ab880ace5797b/users/0c5748d4b4c84292945e7d33641c0c09/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:56.988384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:57.003489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef5e67-a520-42e7-8c0c-2502fbe3cec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a329106baf0d4015b75ab880ace5797b, user_id=0c5748d4b4c84292945e7d33641c0c09, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:57.006215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ef5e67-a520-42e7-8c0c-2502fbe3cec8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:57.019565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1238/2464] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:16:56 2022] PUT /identity/v3/projects/a329106baf0d4015b75ab880ace5797b/users/0c5748d4b4c84292945e7d33641c0c09/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:57.023714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dbb93-bb65-44d5-90d4-cad02f21b557 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:57.034656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320dbb93-bb65-44d5-90d4-cad02f21b557 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:57.055008 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:57.100274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ff19b-2b73-4b05-857a-90a7b8d34253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:57.100884 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ff19b-2b73-4b05-857a-90a7b8d34253 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:57.101161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ff19b-2b73-4b05-857a-90a7b8d34253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:57.101432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ff19b-2b73-4b05-857a-90a7b8d34253 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:57.102114 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ff19b-2b73-4b05-857a-90a7b8d34253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:57.104853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ff19b-2b73-4b05-857a-90a7b8d34253 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:57.115508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1227/2465] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:16:57.120952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef5e67-a520-42e7-8c0c-2502fbe3cec8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:57.131336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ef5e67-a520-42e7-8c0c-2502fbe3cec8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:57.136818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa2336-3ed5-4e68-ab7e-e5a2fc0f71c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:57.137480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa2336-3ed5-4e68-ab7e-e5a2fc0f71c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:57.138048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa2336-3ed5-4e68-ab7e-e5a2fc0f71c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:57.138322 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa2336-3ed5-4e68-ab7e-e5a2fc0f71c3 admin admin] PATH_INFO: `/v3/projects/a329106baf0d4015b75ab880ace5797b/users/0c5748d4b4c84292945e7d33641c0c09/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:57.140235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa2336-3ed5-4e68-ab7e-e5a2fc0f71c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a329106baf0d4015b75ab880ace5797b, user_id=0c5748d4b4c84292945e7d33641c0c09, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:57.142944 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa2336-3ed5-4e68-ab7e-e5a2fc0f71c3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:57.154838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1239/2466] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:16:57 2022] PUT /identity/v3/projects/a329106baf0d4015b75ab880ace5797b/users/0c5748d4b4c84292945e7d33641c0c09/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:57.168093 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54bb39-4dcb-437d-87fc-9dec39b25ed7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:57.168464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54bb39-4dcb-437d-87fc-9dec39b25ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:57.168754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54bb39-4dcb-437d-87fc-9dec39b25ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:57.198502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2a54bb39-4dcb-437d-87fc-9dec39b25ed7 None None] MFA Rules not processed for user `0c5748d4b4c84292945e7d33641c0c09`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:57.199741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2a54bb39-4dcb-437d-87fc-9dec39b25ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:57.282549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1228/2467] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:16:57.294419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa2336-3ed5-4e68-ab7e-e5a2fc0f71c3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:57.296810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-17aa2336-3ed5-4e68-ab7e-e5a2fc0f71c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:57.396290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa2336-3ed5-4e68-ab7e-e5a2fc0f71c3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:57.406527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552d2ab-89ff-4014-bbce-de64c6fd6d5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:57.407168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552d2ab-89ff-4014-bbce-de64c6fd6d5c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:57.407451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552d2ab-89ff-4014-bbce-de64c6fd6d5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:57.407726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552d2ab-89ff-4014-bbce-de64c6fd6d5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:57.422481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552d2ab-89ff-4014-bbce-de64c6fd6d5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:57.424286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552d2ab-89ff-4014-bbce-de64c6fd6d5c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:57.439146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1240/2468] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:16:57 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:16:57.736168 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54bb39-4dcb-437d-87fc-9dec39b25ed7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:57.737675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2a54bb39-4dcb-437d-87fc-9dec39b25ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:57.827121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54bb39-4dcb-437d-87fc-9dec39b25ed7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:57.831646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb2067-4a60-4815-b62d-d92fca5f7627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:57.832327 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb2067-4a60-4815-b62d-d92fca5f7627 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:57.833134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb2067-4a60-4815-b62d-d92fca5f7627 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:57.833458 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb2067-4a60-4815-b62d-d92fca5f7627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:57.836189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3fdb2067-4a60-4815-b62d-d92fca5f7627 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:57.928938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:57.931899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb2067-4a60-4815-b62d-d92fca5f7627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:57.935535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb2067-4a60-4815-b62d-d92fca5f7627 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:57.955362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1229/2469] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:16:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:16:58.168360 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552d2ab-89ff-4014-bbce-de64c6fd6d5c service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:58.253561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552d2ab-89ff-4014-bbce-de64c6fd6d5c service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:58.260442 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:58.261027 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:58.262012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:58.262012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] PATH_INFO: `/v3/users/1e21709066d44a929064a7406dde2fc4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:58.264914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e21709066d44a929064a7406dde2fc4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:58.269033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:58.335467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e21709066d44a929064a7406dde2fc4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:58.336482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e21709066d44a929064a7406dde2fc4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:58.337239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e21709066d44a929064a7406dde2fc4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:58.347650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e21709066d44a929064a7406dde2fc4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:58.349250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:58.365232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e21709066d44a929064a7406dde2fc4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:58.380812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1241/2470] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:16:58 2022] DELETE /identity/v3/users/1e21709066d44a929064a7406dde2fc4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:58.386477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff294f53-6cfb-4aeb-a065-95fc5de8f482 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:58.386758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff294f53-6cfb-4aeb-a065-95fc5de8f482 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:58.387053 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff294f53-6cfb-4aeb-a065-95fc5de8f482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:58.407487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ff294f53-6cfb-4aeb-a065-95fc5de8f482 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:16:58.408714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ff294f53-6cfb-4aeb-a065-95fc5de8f482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:58.488937 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1230/2471] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:16:58.502370 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:58.503921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:58.573329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b2f1d-0a8c-4ae2-8a95-d3b20fb24813 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:58.586763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd2bbf-106b-4edc-8ea9-2949d33a27b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:58.587605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2bbf-106b-4edc-8ea9-2949d33a27b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:58.588007 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2bbf-106b-4edc-8ea9-2949d33a27b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:58.588344 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd2bbf-106b-4edc-8ea9-2949d33a27b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:58.599802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd2bbf-106b-4edc-8ea9-2949d33a27b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:58.601261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd2bbf-106b-4edc-8ea9-2949d33a27b0 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:58.611861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1242/2472] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:16:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:16:59.237514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff294f53-6cfb-4aeb-a065-95fc5de8f482 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:59.238931 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ff294f53-6cfb-4aeb-a065-95fc5de8f482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:16:59.254807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff294f53-6cfb-4aeb-a065-95fc5de8f482 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:59.274428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:59.275681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:59.275961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:59.276238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] PATH_INFO: `/v3/projects/93567d70d1984d0989c855fbfb356095` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:59.278617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93567d70d1984d0989c855fbfb356095)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:59.281440 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:59.343390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93567d70d1984d0989c855fbfb356095', 'request_id': 'req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:59.351996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93567d70d1984d0989c855fbfb356095', 'request_id': 'req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:59.360933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93567d70d1984d0989c855fbfb356095', 'request_id': 'req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:59.379642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] The token cache is being invalidate because project 93567d70d1984d0989c855fbfb356095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:16:59.380043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:59.380900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:59.385714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1231/2473] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:16:59 2022] DELETE /identity/v3/projects/93567d70d1984d0989c855fbfb356095 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:16:59.387072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd2bbf-106b-4edc-8ea9-2949d33a27b0 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:16:59.400391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd2bbf-106b-4edc-8ea9-2949d33a27b0 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:16:59.480743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:16:59.481725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:16:59.481725 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:16:59.481968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] PATH_INFO: `/v3/users/0c5748d4b4c84292945e7d33641c0c09` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:16:59.483400 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5748d4b4c84292945e7d33641c0c09)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:16:59.485126 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:16:59.564281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5748d4b4c84292945e7d33641c0c09'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:59.565416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5748d4b4c84292945e7d33641c0c09'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:59.566263 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5748d4b4c84292945e7d33641c0c09'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:59.578733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5748d4b4c84292945e7d33641c0c09'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:59.604884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3e885b65-67b1-4243-8f67-1e6f3bf28868 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5748d4b4c84292945e7d33641c0c09'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:16:59.611961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:16:59.622174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1243/2474] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:16:59 2022] DELETE /identity/v3/users/0c5748d4b4c84292945e7d33641c0c09 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:00.222699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:00.224747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:00.251997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2afe37-ab43-4f70-b3c2-a3c57ca34ab0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:00.335059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:00.335714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:00.335988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:00.336262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] PATH_INFO: `/v3/projects/a329106baf0d4015b75ab880ace5797b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:00.338273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a329106baf0d4015b75ab880ace5797b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:00.340536 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:00.387324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:00.448736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a329106baf0d4015b75ab880ace5797b', 'request_id': 'req-eed036fb-fd5a-4aaf-ad64-67e565207f5e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:00.464178 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a329106baf0d4015b75ab880ace5797b', 'request_id': 'req-eed036fb-fd5a-4aaf-ad64-67e565207f5e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:00.476726 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a329106baf0d4015b75ab880ace5797b', 'request_id': 'req-eed036fb-fd5a-4aaf-ad64-67e565207f5e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:00.502710 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] The token cache is being invalidate because project a329106baf0d4015b75ab880ace5797b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:17:00.503088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:00.503725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-eed036fb-fd5a-4aaf-ad64-67e565207f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:00.515828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1232/2475] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:17:00 2022] DELETE /identity/v3/projects/a329106baf0d4015b75ab880ace5797b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:00.523304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd104d-198f-446d-b16a-996fd61d48cc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:00.523970 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd104d-198f-446d-b16a-996fd61d48cc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:00.524243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd104d-198f-446d-b16a-996fd61d48cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:00.550033 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-07fd104d-198f-446d-b16a-996fd61d48cc None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:00.551418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-07fd104d-198f-446d-b16a-996fd61d48cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:00.653950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1244/2476] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:17:00.668060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc79e1f-b464-4cd6-ba4d-650496ba5928 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:00.668594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc79e1f-b464-4cd6-ba4d-650496ba5928 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:00.668964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc79e1f-b464-4cd6-ba4d-650496ba5928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:00.708892 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7fc79e1f-b464-4cd6-ba4d-650496ba5928 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:00.710339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7fc79e1f-b464-4cd6-ba4d-650496ba5928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:00.725828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1233/2477] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:00.732831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd104d-198f-446d-b16a-996fd61d48cc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:00.753916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd104d-198f-446d-b16a-996fd61d48cc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:00.767474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce3b37-6212-4e36-a077-893d341b8e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:00.768281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce3b37-6212-4e36-a077-893d341b8e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:00.768566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce3b37-6212-4e36-a077-893d341b8e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:00.768839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce3b37-6212-4e36-a077-893d341b8e6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:00.769508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce3b37-6212-4e36-a077-893d341b8e6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:17:00.769836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce3b37-6212-4e36-a077-893d341b8e6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:00.771490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce3b37-6212-4e36-a077-893d341b8e6e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:00.789656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc79e1f-b464-4cd6-ba4d-650496ba5928 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:00.800675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc79e1f-b464-4cd6-ba4d-650496ba5928 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:00.801383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1245/2478] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:17:00.806385 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bf63d-f0f1-4f9e-8fb5-b3f2c4bc26c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:00.807429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bf63d-f0f1-4f9e-8fb5-b3f2c4bc26c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:00.807714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bf63d-f0f1-4f9e-8fb5-b3f2c4bc26c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:00.807992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bf63d-f0f1-4f9e-8fb5-b3f2c4bc26c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:00.808678 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bf63d-f0f1-4f9e-8fb5-b3f2c4bc26c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:00.810689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bf63d-f0f1-4f9e-8fb5-b3f2c4bc26c6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:00.848861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce3b37-6212-4e36-a077-893d341b8e6e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:00.849652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1234/2479] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:17:00 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:17:00.862055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce3b37-6212-4e36-a077-893d341b8e6e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:00.956014 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fc97c-ea95-48f8-b36f-44e237969c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:00.956659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fc97c-ea95-48f8-b36f-44e237969c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:00.956936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fc97c-ea95-48f8-b36f-44e237969c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:00.957208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fc97c-ea95-48f8-b36f-44e237969c11 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:00.959337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fc97c-ea95-48f8-b36f-44e237969c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:00.964177 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fc97c-ea95-48f8-b36f-44e237969c11 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:01.010905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1246/2480] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:17:00 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:01.013056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bf63d-f0f1-4f9e-8fb5-b3f2c4bc26c6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:01.036439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bf63d-f0f1-4f9e-8fb5-b3f2c4bc26c6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:01.043694 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89f140-b591-4ffa-ba41-5e8280df1abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:01.044387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89f140-b591-4ffa-ba41-5e8280df1abf admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.044599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89f140-b591-4ffa-ba41-5e8280df1abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.044868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff89f140-b591-4ffa-ba41-5e8280df1abf admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.045604 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89f140-b591-4ffa-ba41-5e8280df1abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:01.047265 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff89f140-b591-4ffa-ba41-5e8280df1abf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:01.061005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1235/2481] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:01.066116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fc97c-ea95-48f8-b36f-44e237969c11 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:01.077129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fc97c-ea95-48f8-b36f-44e237969c11 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:01.089438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff603d4-3cf5-4673-8571-f423c43fdac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:01.090184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff603d4-3cf5-4673-8571-f423c43fdac8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.090471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff603d4-3cf5-4673-8571-f423c43fdac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.090751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff603d4-3cf5-4673-8571-f423c43fdac8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.091902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff603d4-3cf5-4673-8571-f423c43fdac8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:01.093236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff603d4-3cf5-4673-8571-f423c43fdac8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:01.108119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1247/2482] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:01.113745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89f140-b591-4ffa-ba41-5e8280df1abf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:01.128418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff89f140-b591-4ffa-ba41-5e8280df1abf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:01.136702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3c571-8791-4765-8c59-6ab2c497c192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:01.136865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3c571-8791-4765-8c59-6ab2c497c192 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.137212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3c571-8791-4765-8c59-6ab2c497c192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.137971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3c571-8791-4765-8c59-6ab2c497c192 admin admin] PATH_INFO: `/v3/projects/95f5d06de0704de486c3155b9da5c540/users/ff8f7f69d4e248969f7b71ac1c1d3df1/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.166914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3c571-8791-4765-8c59-6ab2c497c192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f5d06de0704de486c3155b9da5c540, user_id=ff8f7f69d4e248969f7b71ac1c1d3df1, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:01.170182 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3c571-8791-4765-8c59-6ab2c497c192 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:01.189830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1236/2483] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:17:01 2022] PUT /identity/v3/projects/95f5d06de0704de486c3155b9da5c540/users/ff8f7f69d4e248969f7b71ac1c1d3df1/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:01.199619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ba3ee-a41f-4a1e-b595-ca51e31d91ae None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.199909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ba3ee-a41f-4a1e-b595-ca51e31d91ae None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.200186 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ba3ee-a41f-4a1e-b595-ca51e31d91ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.232300 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-966ba3ee-a41f-4a1e-b595-ca51e31d91ae None None] MFA Rules not processed for user `ff8f7f69d4e248969f7b71ac1c1d3df1`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:01.233573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-966ba3ee-a41f-4a1e-b595-ca51e31d91ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:01.293001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:01.320714 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1248/2484] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:01.325814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3c571-8791-4765-8c59-6ab2c497c192 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:01.395963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3c571-8791-4765-8c59-6ab2c497c192 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:01.401226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cea90e-1238-46c0-b677-cbacb4d666c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:01.401882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cea90e-1238-46c0-b677-cbacb4d666c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.402167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cea90e-1238-46c0-b677-cbacb4d666c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.402448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cea90e-1238-46c0-b677-cbacb4d666c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.403233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cea90e-1238-46c0-b677-cbacb4d666c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:01.405087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cea90e-1238-46c0-b677-cbacb4d666c3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:01.438498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1237/2485] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:17:01 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:01.445009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ba3ee-a41f-4a1e-b595-ca51e31d91ae None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:01.457851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ba3ee-a41f-4a1e-b595-ca51e31d91ae None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:01.527762 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74a5b3-d9c1-4993-b1a3-4813d57fca61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:01.528422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74a5b3-d9c1-4993-b1a3-4813d57fca61 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.528701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74a5b3-d9c1-4993-b1a3-4813d57fca61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.528967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74a5b3-d9c1-4993-b1a3-4813d57fca61 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.529600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74a5b3-d9c1-4993-b1a3-4813d57fca61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:01.531825 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74a5b3-d9c1-4993-b1a3-4813d57fca61 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:01.562464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1249/2486] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:17:01 2022] POST /identity/v3/users => generated 297 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:01.566735 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cea90e-1238-46c0-b677-cbacb4d666c3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:01.579618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cea90e-1238-46c0-b677-cbacb4d666c3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:01.584684 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1dd7c6-6891-4399-9f2a-9794b2597334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:01.585296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dd7c6-6891-4399-9f2a-9794b2597334 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.585570 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dd7c6-6891-4399-9f2a-9794b2597334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.585869 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dd7c6-6891-4399-9f2a-9794b2597334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.586526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1dd7c6-6891-4399-9f2a-9794b2597334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:01.589374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1dd7c6-6891-4399-9f2a-9794b2597334 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:01.601214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1238/2487] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:01.605813 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74a5b3-d9c1-4993-b1a3-4813d57fca61 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:01.617200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74a5b3-d9c1-4993-b1a3-4813d57fca61 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:01.622928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe6b8-9af7-4534-af04-c5784036bdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:01.625343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe6b8-9af7-4534-af04-c5784036bdbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.625343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe6b8-9af7-4534-af04-c5784036bdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.625343 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881fe6b8-9af7-4534-af04-c5784036bdbb admin admin] PATH_INFO: `/v3/projects/7967d036b29e41eaadc19824cbbd6abf/users/75a4d8f776b04035aed662cc5dbb42fb/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.641708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fe6b8-9af7-4534-af04-c5784036bdbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7967d036b29e41eaadc19824cbbd6abf, user_id=75a4d8f776b04035aed662cc5dbb42fb, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:01.646624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881fe6b8-9af7-4534-af04-c5784036bdbb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:01.665233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1250/2488] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:17:01 2022] PUT /identity/v3/projects/7967d036b29e41eaadc19824cbbd6abf/users/75a4d8f776b04035aed662cc5dbb42fb/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:17:01.670920 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1dd7c6-6891-4399-9f2a-9794b2597334 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:01.689045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1dd7c6-6891-4399-9f2a-9794b2597334 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:01.713368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:01.760678 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88def6bf-80f9-4df0-86c6-3094bc89c679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:01.761199 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88def6bf-80f9-4df0-86c6-3094bc89c679 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.761473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88def6bf-80f9-4df0-86c6-3094bc89c679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.761746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88def6bf-80f9-4df0-86c6-3094bc89c679 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.762957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88def6bf-80f9-4df0-86c6-3094bc89c679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:01.764315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88def6bf-80f9-4df0-86c6-3094bc89c679 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:01.775024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1239/2489] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:01.779370 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe6b8-9af7-4534-af04-c5784036bdbb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:01.800539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881fe6b8-9af7-4534-af04-c5784036bdbb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:01.807414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576f01-aead-4397-b5b2-0530681b2477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:01.807724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576f01-aead-4397-b5b2-0530681b2477 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.807889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576f01-aead-4397-b5b2-0530681b2477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.808086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba576f01-aead-4397-b5b2-0530681b2477 admin admin] PATH_INFO: `/v3/projects/7967d036b29e41eaadc19824cbbd6abf/users/75a4d8f776b04035aed662cc5dbb42fb/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.810216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576f01-aead-4397-b5b2-0530681b2477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7967d036b29e41eaadc19824cbbd6abf, user_id=75a4d8f776b04035aed662cc5dbb42fb, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:01.812907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba576f01-aead-4397-b5b2-0530681b2477 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:01.825585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1251/2490] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:17:01 2022] PUT /identity/v3/projects/7967d036b29e41eaadc19824cbbd6abf/users/75a4d8f776b04035aed662cc5dbb42fb/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:01.895292 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59bcde-1139-4bb0-ae34-387fa01a447a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:01.895600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59bcde-1139-4bb0-ae34-387fa01a447a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:01.895880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59bcde-1139-4bb0-ae34-387fa01a447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:01.938521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3c59bcde-1139-4bb0-ae34-387fa01a447a None None] MFA Rules not processed for user `75a4d8f776b04035aed662cc5dbb42fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:01.939991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c59bcde-1139-4bb0-ae34-387fa01a447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:02.023849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1240/2491] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:02.035622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576f01-aead-4397-b5b2-0530681b2477 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:02.037413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ba576f01-aead-4397-b5b2-0530681b2477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:02.111163 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba576f01-aead-4397-b5b2-0530681b2477 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:02.115034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debbddc-715e-4281-8989-89de16109cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:02.115654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debbddc-715e-4281-8989-89de16109cad service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:02.115926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debbddc-715e-4281-8989-89de16109cad service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:02.116198 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debbddc-715e-4281-8989-89de16109cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:02.129226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0debbddc-715e-4281-8989-89de16109cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:02.130730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0debbddc-715e-4281-8989-89de16109cad service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:02.142709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1252/2492] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:17:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:17:02.566629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59bcde-1139-4bb0-ae34-387fa01a447a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:02.760270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59bcde-1139-4bb0-ae34-387fa01a447a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:02.765135 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:02.765783 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:02.766064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:02.766334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] PATH_INFO: `/v3/users/ff8f7f69d4e248969f7b71ac1c1d3df1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:02.767456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff8f7f69d4e248969f7b71ac1c1d3df1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:02.769381 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:02.817728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:02.822220 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff8f7f69d4e248969f7b71ac1c1d3df1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:02.832539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff8f7f69d4e248969f7b71ac1c1d3df1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:02.833278 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff8f7f69d4e248969f7b71ac1c1d3df1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:02.833930 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff8f7f69d4e248969f7b71ac1c1d3df1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:02.842297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff8f7f69d4e248969f7b71ac1c1d3df1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:02.860926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1241/2493] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:17:02 2022] DELETE /identity/v3/users/ff8f7f69d4e248969f7b71ac1c1d3df1 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:17:02.866143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457eec7-9325-46d7-ad47-43ac43fb5ea7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:02.866659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457eec7-9325-46d7-ad47-43ac43fb5ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:02.866760 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457eec7-9325-46d7-ad47-43ac43fb5ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:02.887005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7457eec7-9325-46d7-ad47-43ac43fb5ea7 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:02.888503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7457eec7-9325-46d7-ad47-43ac43fb5ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:02.930754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:02.952915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1253/2494] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:02.963479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:02.965005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:03.025251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c9b5a-0b9c-4acf-8e1d-0d3029be32cd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:03.029060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2081d7-03a1-469c-a658-66c8a809b744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:03.029682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2081d7-03a1-469c-a658-66c8a809b744 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:03.029994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2081d7-03a1-469c-a658-66c8a809b744 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:03.030270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2081d7-03a1-469c-a658-66c8a809b744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:03.041220 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2081d7-03a1-469c-a658-66c8a809b744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:03.044294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2081d7-03a1-469c-a658-66c8a809b744 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:03.059102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1242/2495] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:17:03.375612 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457eec7-9325-46d7-ad47-43ac43fb5ea7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:03.377168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7457eec7-9325-46d7-ad47-43ac43fb5ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:03.402266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457eec7-9325-46d7-ad47-43ac43fb5ea7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:03.412514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:03.413928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:03.414346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:03.414697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] PATH_INFO: `/v3/projects/95f5d06de0704de486c3155b9da5c540` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:03.416181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f5d06de0704de486c3155b9da5c540)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:03.418241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:03.481545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f5d06de0704de486c3155b9da5c540', 'request_id': 'req-743cf7a8-81a9-4f89-91cf-732a0ae54e34'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:03.491761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f5d06de0704de486c3155b9da5c540', 'request_id': 'req-743cf7a8-81a9-4f89-91cf-732a0ae54e34'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:03.498855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f5d06de0704de486c3155b9da5c540', 'request_id': 'req-743cf7a8-81a9-4f89-91cf-732a0ae54e34'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:03.514610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] The token cache is being invalidate because project 95f5d06de0704de486c3155b9da5c540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:17:03.514912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:03.515861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:03.517873 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1254/2496] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:17:03 2022] DELETE /identity/v3/projects/95f5d06de0704de486c3155b9da5c540 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:03.522428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2081d7-03a1-469c-a658-66c8a809b744 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:03.534618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2081d7-03a1-469c-a658-66c8a809b744 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:03.616295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:03.617056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:03.617420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:03.617758 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] PATH_INFO: `/v3/users/75a4d8f776b04035aed662cc5dbb42fb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:03.619480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=75a4d8f776b04035aed662cc5dbb42fb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:03.621453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:03.695182 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75a4d8f776b04035aed662cc5dbb42fb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:03.703645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75a4d8f776b04035aed662cc5dbb42fb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:03.705277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75a4d8f776b04035aed662cc5dbb42fb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:03.705941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75a4d8f776b04035aed662cc5dbb42fb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:03.716022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4369aced-391c-4081-8885-499dc6643d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75a4d8f776b04035aed662cc5dbb42fb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:03.718547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:03.746872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1243/2497] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:17:03 2022] DELETE /identity/v3/users/75a4d8f776b04035aed662cc5dbb42fb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:04.073523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:04.075178 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:04.114819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cf7a8-81a9-4f89-91cf-732a0ae54e34 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:04.218193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:04.219079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:04.219475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:04.219861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] PATH_INFO: `/v3/projects/7967d036b29e41eaadc19824cbbd6abf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:04.221305 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7967d036b29e41eaadc19824cbbd6abf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:04.224663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:04.294180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7967d036b29e41eaadc19824cbbd6abf', 'request_id': 'req-b2c904cc-2870-415e-8e1e-803448d5d90e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:04.304771 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7967d036b29e41eaadc19824cbbd6abf', 'request_id': 'req-b2c904cc-2870-415e-8e1e-803448d5d90e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:04.313872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7967d036b29e41eaadc19824cbbd6abf', 'request_id': 'req-b2c904cc-2870-415e-8e1e-803448d5d90e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:04.327002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:04.334440 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] The token cache is being invalidate because project 7967d036b29e41eaadc19824cbbd6abf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:17:04.334922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:04.335846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2c904cc-2870-415e-8e1e-803448d5d90e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:04.339947 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1255/2498] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:17:04 2022] DELETE /identity/v3/projects/7967d036b29e41eaadc19824cbbd6abf => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:04.345446 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac99df-ccee-4107-b3a9-219a5a31805c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:04.345838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac99df-ccee-4107-b3a9-219a5a31805c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:04.346051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac99df-ccee-4107-b3a9-219a5a31805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:04.370003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dfac99df-ccee-4107-b3a9-219a5a31805c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:04.371154 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dfac99df-ccee-4107-b3a9-219a5a31805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:04.436542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1244/2499] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:04.444585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d13e-32ac-410e-b7df-c0a58ae7f346 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:04.444912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d13e-32ac-410e-b7df-c0a58ae7f346 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:04.445238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d13e-32ac-410e-b7df-c0a58ae7f346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:04.471233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5a98d13e-32ac-410e-b7df-c0a58ae7f346 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:04.472640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5a98d13e-32ac-410e-b7df-c0a58ae7f346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:04.482138 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1256/2500] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:04.486802 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac99df-ccee-4107-b3a9-219a5a31805c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:04.499194 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac99df-ccee-4107-b3a9-219a5a31805c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:04.504128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138c9ef-127f-42b0-a0fd-bc4be0841923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:04.504752 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138c9ef-127f-42b0-a0fd-bc4be0841923 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:04.505029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138c9ef-127f-42b0-a0fd-bc4be0841923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:04.505299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138c9ef-127f-42b0-a0fd-bc4be0841923 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:04.505986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138c9ef-127f-42b0-a0fd-bc4be0841923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:17:04.506288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138c9ef-127f-42b0-a0fd-bc4be0841923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:04.507926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f138c9ef-127f-42b0-a0fd-bc4be0841923 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:04.526315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1245/2501] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:17:04.530472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d13e-32ac-410e-b7df-c0a58ae7f346 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:04.542704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d13e-32ac-410e-b7df-c0a58ae7f346 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:04.550513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fa0b6-e194-43b1-993d-aaed0e8907ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:04.551398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fa0b6-e194-43b1-993d-aaed0e8907ba admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:04.551746 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fa0b6-e194-43b1-993d-aaed0e8907ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:04.552100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fa0b6-e194-43b1-993d-aaed0e8907ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:04.552818 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fa0b6-e194-43b1-993d-aaed0e8907ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:04.555040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165fa0b6-e194-43b1-993d-aaed0e8907ba admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:04.588465 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1257/2502] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:17:04 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:04.593630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138c9ef-127f-42b0-a0fd-bc4be0841923 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:04.605591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f138c9ef-127f-42b0-a0fd-bc4be0841923 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:04.686951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f76f4-829e-478f-8ec1-6d242179dd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:04.689942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f76f4-829e-478f-8ec1-6d242179dd58 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:04.690493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f76f4-829e-478f-8ec1-6d242179dd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:04.691402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f76f4-829e-478f-8ec1-6d242179dd58 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:04.692396 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f76f4-829e-478f-8ec1-6d242179dd58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:04.698079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f76f4-829e-478f-8ec1-6d242179dd58 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:04.736955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1246/2503] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:17:04 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:17:04.741259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fa0b6-e194-43b1-993d-aaed0e8907ba admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:04.757023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165fa0b6-e194-43b1-993d-aaed0e8907ba admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:04.767244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b65265-c60b-49d8-a659-a7492a82a4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:04.768069 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b65265-c60b-49d8-a659-a7492a82a4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:04.768413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b65265-c60b-49d8-a659-a7492a82a4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:04.768761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b65265-c60b-49d8-a659-a7492a82a4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:04.769574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b65265-c60b-49d8-a659-a7492a82a4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:04.771298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b65265-c60b-49d8-a659-a7492a82a4e7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:04.784304 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1258/2504] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:04.787984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f76f4-829e-478f-8ec1-6d242179dd58 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:04.804497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f76f4-829e-478f-8ec1-6d242179dd58 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:04.810627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943de688-fab7-48ca-9bfb-83c52eb3334e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:04.811309 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de688-fab7-48ca-9bfb-83c52eb3334e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:04.811653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de688-fab7-48ca-9bfb-83c52eb3334e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:04.812016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de688-fab7-48ca-9bfb-83c52eb3334e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:04.812749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943de688-fab7-48ca-9bfb-83c52eb3334e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:04.814744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943de688-fab7-48ca-9bfb-83c52eb3334e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:04.827681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1247/2505] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:04.831754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b65265-c60b-49d8-a659-a7492a82a4e7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:04.849467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b65265-c60b-49d8-a659-a7492a82a4e7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:04.854538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279441f-8d9f-4bb4-8140-57a9b9a5c005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:04.855233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279441f-8d9f-4bb4-8140-57a9b9a5c005 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:04.855515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279441f-8d9f-4bb4-8140-57a9b9a5c005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:04.855795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279441f-8d9f-4bb4-8140-57a9b9a5c005 admin admin] PATH_INFO: `/v3/projects/116ea9af34394562a5fc817e2e780f48/users/771435fabff94794876f63d3edb0a5f2/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:04.872371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279441f-8d9f-4bb4-8140-57a9b9a5c005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116ea9af34394562a5fc817e2e780f48, user_id=771435fabff94794876f63d3edb0a5f2, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:04.875102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279441f-8d9f-4bb4-8140-57a9b9a5c005 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:04.894071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1259/2506] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:17:04 2022] PUT /identity/v3/projects/116ea9af34394562a5fc817e2e780f48/users/771435fabff94794876f63d3edb0a5f2/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:04.906227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e79f9-c1af-458b-a2ee-4f5bd383297a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:04.906522 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e79f9-c1af-458b-a2ee-4f5bd383297a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:04.906851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e79f9-c1af-458b-a2ee-4f5bd383297a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:04.936287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e08e79f9-c1af-458b-a2ee-4f5bd383297a None None] MFA Rules not processed for user `771435fabff94794876f63d3edb0a5f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:04.937521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e08e79f9-c1af-458b-a2ee-4f5bd383297a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:04.985830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:05.019189 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1248/2507] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:05.030819 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279441f-8d9f-4bb4-8140-57a9b9a5c005 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:05.032697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b279441f-8d9f-4bb4-8140-57a9b9a5c005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:05.112140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279441f-8d9f-4bb4-8140-57a9b9a5c005 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:05.115917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ee466-e960-4b0d-80fc-d0d3dc8d3172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:05.116538 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ee466-e960-4b0d-80fc-d0d3dc8d3172 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:05.117156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ee466-e960-4b0d-80fc-d0d3dc8d3172 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:05.117444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ee466-e960-4b0d-80fc-d0d3dc8d3172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:05.128284 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ee466-e960-4b0d-80fc-d0d3dc8d3172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:05.129824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ee466-e960-4b0d-80fc-d0d3dc8d3172 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:05.140884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1260/2508] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:17:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:17:05.161353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e79f9-c1af-458b-a2ee-4f5bd383297a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:05.231791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e79f9-c1af-458b-a2ee-4f5bd383297a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:05.236629 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:05.237254 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:05.237533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:05.237838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] PATH_INFO: `/v3/users/771435fabff94794876f63d3edb0a5f2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:05.239005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] RBAC: Authorizing `identity:delete_user(user_id=771435fabff94794876f63d3edb0a5f2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:05.240970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:05.299359 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '771435fabff94794876f63d3edb0a5f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:05.307579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '771435fabff94794876f63d3edb0a5f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:05.308181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '771435fabff94794876f63d3edb0a5f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:05.308802 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '771435fabff94794876f63d3edb0a5f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:05.315806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '771435fabff94794876f63d3edb0a5f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:05.333040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1249/2509] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:17:05 2022] DELETE /identity/v3/users/771435fabff94794876f63d3edb0a5f2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:05.341821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d05d06-e7c2-489b-b5dd-9d3bb12da6d0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:05.342109 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d05d06-e7c2-489b-b5dd-9d3bb12da6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:05.342387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d05d06-e7c2-489b-b5dd-9d3bb12da6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:05.361698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-94d05d06-e7c2-489b-b5dd-9d3bb12da6d0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:05.362956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-94d05d06-e7c2-489b-b5dd-9d3bb12da6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:05.431689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1261/2510] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:05.441567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:05.443045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:05.511349 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b1183-e085-49e5-afa9-e2d5d5f82ccc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:05.515513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62babe21-15ab-4192-9ed4-d387958cc9c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:05.516124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62babe21-15ab-4192-9ed4-d387958cc9c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:05.516399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62babe21-15ab-4192-9ed4-d387958cc9c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:05.516668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62babe21-15ab-4192-9ed4-d387958cc9c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:05.527635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62babe21-15ab-4192-9ed4-d387958cc9c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:05.529071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62babe21-15ab-4192-9ed4-d387958cc9c9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:05.534264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:05.542002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1250/2511] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:17:05.856790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d05d06-e7c2-489b-b5dd-9d3bb12da6d0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:05.856790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-94d05d06-e7c2-489b-b5dd-9d3bb12da6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:05.869314 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d05d06-e7c2-489b-b5dd-9d3bb12da6d0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:05.875396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:05.876203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:05.876587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:05.877395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] PATH_INFO: `/v3/projects/116ea9af34394562a5fc817e2e780f48` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:05.878591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] RBAC: Authorizing `identity:delete_project(project_id=116ea9af34394562a5fc817e2e780f48)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:05.881030 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:05.938909 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:05.959193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116ea9af34394562a5fc817e2e780f48', 'request_id': 'req-aa018852-db85-42c8-8669-7820bcacd49b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:05.969624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116ea9af34394562a5fc817e2e780f48', 'request_id': 'req-aa018852-db85-42c8-8669-7820bcacd49b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:05.976049 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116ea9af34394562a5fc817e2e780f48', 'request_id': 'req-aa018852-db85-42c8-8669-7820bcacd49b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:05.989197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] The token cache is being invalidate because project 116ea9af34394562a5fc817e2e780f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:17:05.989502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:05.990203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa018852-db85-42c8-8669-7820bcacd49b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:05.991717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1262/2512] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:17:05 2022] DELETE /identity/v3/projects/116ea9af34394562a5fc817e2e780f48 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:05.997392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc3aea-ccaf-4e3a-9ca1-48c8daa55afc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:05.997957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc3aea-ccaf-4e3a-9ca1-48c8daa55afc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:05.997957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dc3aea-ccaf-4e3a-9ca1-48c8daa55afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.018989 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-84dc3aea-ccaf-4e3a-9ca1-48c8daa55afc None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:06.020241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-84dc3aea-ccaf-4e3a-9ca1-48c8daa55afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:06.122346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6f0e9-4c62-468e-bccf-e48ad11d7e06 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:06.123438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6f0e9-4c62-468e-bccf-e48ad11d7e06 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:06.123438 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c6f0e9-4c62-468e-bccf-e48ad11d7e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.129549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1251/2513] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:17:06.146501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-76c6f0e9-4c62-468e-bccf-e48ad11d7e06 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:06.147769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-76c6f0e9-4c62-468e-bccf-e48ad11d7e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:06.157424 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1263/2514] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:06.163083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc3aea-ccaf-4e3a-9ca1-48c8daa55afc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:06.174403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dc3aea-ccaf-4e3a-9ca1-48c8daa55afc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:06.179163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf1cca-93f9-4c55-9b52-27e1715b858b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:06.179771 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf1cca-93f9-4c55-9b52-27e1715b858b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:06.180062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf1cca-93f9-4c55-9b52-27e1715b858b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:06.180313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf1cca-93f9-4c55-9b52-27e1715b858b admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.181005 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf1cca-93f9-4c55-9b52-27e1715b858b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:17:06.181306 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf1cca-93f9-4c55-9b52-27e1715b858b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:06.183066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf1cca-93f9-4c55-9b52-27e1715b858b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:06.197271 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1252/2515] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:17:06.202532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6f0e9-4c62-468e-bccf-e48ad11d7e06 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:06.218557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c6f0e9-4c62-468e-bccf-e48ad11d7e06 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:06.223236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29e78a-1085-404c-90ea-7fd7fedc9a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:06.223870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29e78a-1085-404c-90ea-7fd7fedc9a48 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:06.224183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29e78a-1085-404c-90ea-7fd7fedc9a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:06.224407 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29e78a-1085-404c-90ea-7fd7fedc9a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.225046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29e78a-1085-404c-90ea-7fd7fedc9a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:06.227026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29e78a-1085-404c-90ea-7fd7fedc9a48 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:06.261828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1264/2516] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:17:06 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:06.265650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf1cca-93f9-4c55-9b52-27e1715b858b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:06.278451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf1cca-93f9-4c55-9b52-27e1715b858b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:06.348393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beea9b-46c6-4f53-897a-cb4b1165aecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:06.349016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beea9b-46c6-4f53-897a-cb4b1165aecd admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:06.349295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beea9b-46c6-4f53-897a-cb4b1165aecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:06.349568 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39beea9b-46c6-4f53-897a-cb4b1165aecd admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.350233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beea9b-46c6-4f53-897a-cb4b1165aecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:06.352161 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39beea9b-46c6-4f53-897a-cb4b1165aecd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:06.389123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1253/2517] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:17:06 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:06.390598 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29e78a-1085-404c-90ea-7fd7fedc9a48 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:06.404314 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29e78a-1085-404c-90ea-7fd7fedc9a48 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:06.411016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9cc4e-4c0b-46bb-b6e7-971d13ea07da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:06.411651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9cc4e-4c0b-46bb-b6e7-971d13ea07da admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:06.411926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9cc4e-4c0b-46bb-b6e7-971d13ea07da admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:06.412194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9cc4e-4c0b-46bb-b6e7-971d13ea07da admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.412850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9cc4e-4c0b-46bb-b6e7-971d13ea07da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:06.414505 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9cc4e-4c0b-46bb-b6e7-971d13ea07da admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:06.427061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1265/2518] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:06.431661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beea9b-46c6-4f53-897a-cb4b1165aecd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:06.448795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39beea9b-46c6-4f53-897a-cb4b1165aecd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:06.453757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b62c94-1b61-4ba1-be60-ffd093d24aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:06.454404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b62c94-1b61-4ba1-be60-ffd093d24aad admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:06.454703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b62c94-1b61-4ba1-be60-ffd093d24aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:06.454959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b62c94-1b61-4ba1-be60-ffd093d24aad admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.455609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b62c94-1b61-4ba1-be60-ffd093d24aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:06.457220 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b62c94-1b61-4ba1-be60-ffd093d24aad admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:06.470374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1254/2519] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:17:06.474508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9cc4e-4c0b-46bb-b6e7-971d13ea07da admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:06.489572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9cc4e-4c0b-46bb-b6e7-971d13ea07da admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:06.495467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb217dd-f2ac-4202-becb-8c4ee05e35be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:06.496165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb217dd-f2ac-4202-becb-8c4ee05e35be admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:06.496468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb217dd-f2ac-4202-becb-8c4ee05e35be admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:06.496748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb217dd-f2ac-4202-becb-8c4ee05e35be admin admin] PATH_INFO: `/v3/projects/07be24332ca94de3ae76838813b3b102/users/a9cc2890a15f412b8adaee9e80953763/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.518519 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb217dd-f2ac-4202-becb-8c4ee05e35be admin admin] RBAC: Authorizing `identity:create_grant(project_id=07be24332ca94de3ae76838813b3b102, user_id=a9cc2890a15f412b8adaee9e80953763, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:06.521797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb217dd-f2ac-4202-becb-8c4ee05e35be admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:06.546643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1266/2520] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:17:06 2022] PUT /identity/v3/projects/07be24332ca94de3ae76838813b3b102/users/a9cc2890a15f412b8adaee9e80953763/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:06.557751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10e1c-8485-4176-8268-5986dbab6390 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:06.558188 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10e1c-8485-4176-8268-5986dbab6390 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:06.558378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e10e1c-8485-4176-8268-5986dbab6390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.590169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-99e10e1c-8485-4176-8268-5986dbab6390 None None] MFA Rules not processed for user `a9cc2890a15f412b8adaee9e80953763`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:06.591412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-99e10e1c-8485-4176-8268-5986dbab6390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:06.658742 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:06.671241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1255/2521] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:06.675959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb217dd-f2ac-4202-becb-8c4ee05e35be admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:06.763499 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb217dd-f2ac-4202-becb-8c4ee05e35be admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:06.768644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a7f4f-d455-4dd7-be1f-2749708a2f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:06.769282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a7f4f-d455-4dd7-be1f-2749708a2f5e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:06.769588 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a7f4f-d455-4dd7-be1f-2749708a2f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:06.769880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a7f4f-d455-4dd7-be1f-2749708a2f5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.770528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a7f4f-d455-4dd7-be1f-2749708a2f5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:06.772487 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a7f4f-d455-4dd7-be1f-2749708a2f5e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:06.810207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1267/2522] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:17:06 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:06.816792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e10e1c-8485-4176-8268-5986dbab6390 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:06.832509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e10e1c-8485-4176-8268-5986dbab6390 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:06.933013 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41a12d-d8de-46ad-9ee6-3dbffab9a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:06.933641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41a12d-d8de-46ad-9ee6-3dbffab9a85a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:06.933957 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41a12d-d8de-46ad-9ee6-3dbffab9a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:06.934233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41a12d-d8de-46ad-9ee6-3dbffab9a85a admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:06.935257 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41a12d-d8de-46ad-9ee6-3dbffab9a85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:06.937279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41a12d-d8de-46ad-9ee6-3dbffab9a85a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:06.975543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1256/2523] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:17:06 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:06.981233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a7f4f-d455-4dd7-be1f-2749708a2f5e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:06.994700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a7f4f-d455-4dd7-be1f-2749708a2f5e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:07.000946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cc267-5e27-4c5a-a6b2-6022ffbed0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:07.001624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cc267-5e27-4c5a-a6b2-6022ffbed0aa admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.002086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cc267-5e27-4c5a-a6b2-6022ffbed0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.002642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cc267-5e27-4c5a-a6b2-6022ffbed0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.003777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cc267-5e27-4c5a-a6b2-6022ffbed0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:07.005843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cc267-5e27-4c5a-a6b2-6022ffbed0aa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:07.025550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1268/2524] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:07.028018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41a12d-d8de-46ad-9ee6-3dbffab9a85a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:07.038400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41a12d-d8de-46ad-9ee6-3dbffab9a85a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:07.043589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675c6b7-8381-4cb1-88db-9385bd137975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:07.044340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675c6b7-8381-4cb1-88db-9385bd137975 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.044679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675c6b7-8381-4cb1-88db-9385bd137975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.044968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675c6b7-8381-4cb1-88db-9385bd137975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.045644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675c6b7-8381-4cb1-88db-9385bd137975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:07.047547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675c6b7-8381-4cb1-88db-9385bd137975 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:07.066035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1257/2525] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:17:07.067820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cc267-5e27-4c5a-a6b2-6022ffbed0aa admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:07.080776 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cc267-5e27-4c5a-a6b2-6022ffbed0aa admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:07.091286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09337e0-234c-4138-94e7-eca0e20be050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:07.092082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09337e0-234c-4138-94e7-eca0e20be050 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.092412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09337e0-234c-4138-94e7-eca0e20be050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.092739 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09337e0-234c-4138-94e7-eca0e20be050 admin admin] PATH_INFO: `/v3/projects/4936ba803cd7455fa8cbe200034ee3a3/users/9cecfea5d5a5453fb2757fc3adce9835/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.118710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09337e0-234c-4138-94e7-eca0e20be050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4936ba803cd7455fa8cbe200034ee3a3, user_id=9cecfea5d5a5453fb2757fc3adce9835, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:07.121590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09337e0-234c-4138-94e7-eca0e20be050 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:07.148371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1269/2526] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:17:07 2022] PUT /identity/v3/projects/4936ba803cd7455fa8cbe200034ee3a3/users/9cecfea5d5a5453fb2757fc3adce9835/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:07.160307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608a2e8-d1ca-4b13-a20c-e084598da84a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.160593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608a2e8-d1ca-4b13-a20c-e084598da84a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.160929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608a2e8-d1ca-4b13-a20c-e084598da84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.193004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d608a2e8-d1ca-4b13-a20c-e084598da84a None None] MFA Rules not processed for user `9cecfea5d5a5453fb2757fc3adce9835`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:07.194311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d608a2e8-d1ca-4b13-a20c-e084598da84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:07.292946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1258/2527] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:07.298017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09337e0-234c-4138-94e7-eca0e20be050 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:07.310851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:07.384885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09337e0-234c-4138-94e7-eca0e20be050 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:07.390384 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2199b7e-f040-4d32-8e02-dab59db651af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:07.391028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2199b7e-f040-4d32-8e02-dab59db651af admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.391308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2199b7e-f040-4d32-8e02-dab59db651af admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.391574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2199b7e-f040-4d32-8e02-dab59db651af admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.392226 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2199b7e-f040-4d32-8e02-dab59db651af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:07.394192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2199b7e-f040-4d32-8e02-dab59db651af admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:07.438849 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1270/2528] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:17:07 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:07.444413 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608a2e8-d1ca-4b13-a20c-e084598da84a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:07.457067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608a2e8-d1ca-4b13-a20c-e084598da84a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:07.522948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b89c6-0f15-48e7-b47d-68949ed83507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:07.523578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b89c6-0f15-48e7-b47d-68949ed83507 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.523875 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b89c6-0f15-48e7-b47d-68949ed83507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.524138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b89c6-0f15-48e7-b47d-68949ed83507 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.524827 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b89c6-0f15-48e7-b47d-68949ed83507 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:07.527232 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b89c6-0f15-48e7-b47d-68949ed83507 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:07.542747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:07.557320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1259/2529] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:17:07 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:17:07.561124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2199b7e-f040-4d32-8e02-dab59db651af admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:07.574368 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2199b7e-f040-4d32-8e02-dab59db651af admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:07.579488 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac4e71-8458-4989-a2e0-c8e9ef5c8264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:07.580119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac4e71-8458-4989-a2e0-c8e9ef5c8264 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.580398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac4e71-8458-4989-a2e0-c8e9ef5c8264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.580669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac4e71-8458-4989-a2e0-c8e9ef5c8264 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.581327 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac4e71-8458-4989-a2e0-c8e9ef5c8264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:07.583015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac4e71-8458-4989-a2e0-c8e9ef5c8264 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:07.596631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1271/2530] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:07.601659 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b89c6-0f15-48e7-b47d-68949ed83507 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:07.615203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b89c6-0f15-48e7-b47d-68949ed83507 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:07.621130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01594f09-2d71-43ae-a1af-8149e9bb9058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:07.621833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594f09-2d71-43ae-a1af-8149e9bb9058 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.622113 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594f09-2d71-43ae-a1af-8149e9bb9058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.622387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01594f09-2d71-43ae-a1af-8149e9bb9058 admin admin] PATH_INFO: `/v3/projects/33267721b8fb44fe877f680c8b609b7e/users/7b1233c400554fc4b80a8c76f491d743/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.640448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01594f09-2d71-43ae-a1af-8149e9bb9058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33267721b8fb44fe877f680c8b609b7e, user_id=7b1233c400554fc4b80a8c76f491d743, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:07.643240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01594f09-2d71-43ae-a1af-8149e9bb9058 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:07.664224 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1260/2531] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:17:07 2022] PUT /identity/v3/projects/33267721b8fb44fe877f680c8b609b7e/users/7b1233c400554fc4b80a8c76f491d743/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:07.669117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac4e71-8458-4989-a2e0-c8e9ef5c8264 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:07.679717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac4e71-8458-4989-a2e0-c8e9ef5c8264 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:07.766715 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85881ec-8901-4c2f-9bc5-7cf08655e3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:07.767344 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85881ec-8901-4c2f-9bc5-7cf08655e3ee admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.767610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85881ec-8901-4c2f-9bc5-7cf08655e3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.767890 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85881ec-8901-4c2f-9bc5-7cf08655e3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.768536 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85881ec-8901-4c2f-9bc5-7cf08655e3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:07.770192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85881ec-8901-4c2f-9bc5-7cf08655e3ee admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:07.784056 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1272/2532] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:07.788233 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01594f09-2d71-43ae-a1af-8149e9bb9058 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:07.800084 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01594f09-2d71-43ae-a1af-8149e9bb9058 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:07.804855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d1a0f-5e49-4faa-a620-1475946389ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:07.805524 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d1a0f-5e49-4faa-a620-1475946389ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.806448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d1a0f-5e49-4faa-a620-1475946389ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.806448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d1a0f-5e49-4faa-a620-1475946389ef admin admin] PATH_INFO: `/v3/projects/33267721b8fb44fe877f680c8b609b7e/users/7b1233c400554fc4b80a8c76f491d743/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.808124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d1a0f-5e49-4faa-a620-1475946389ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=33267721b8fb44fe877f680c8b609b7e, user_id=7b1233c400554fc4b80a8c76f491d743, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:07.810900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d1a0f-5e49-4faa-a620-1475946389ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:07.824537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1261/2533] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:17:07 2022] PUT /identity/v3/projects/33267721b8fb44fe877f680c8b609b7e/users/7b1233c400554fc4b80a8c76f491d743/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:07.836416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39aa9c-1828-4e58-b275-9a301d0f9cfd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:07.836633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39aa9c-1828-4e58-b275-9a301d0f9cfd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:07.837522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39aa9c-1828-4e58-b275-9a301d0f9cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:07.871375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6c39aa9c-1828-4e58-b275-9a301d0f9cfd None None] MFA Rules not processed for user `7b1233c400554fc4b80a8c76f491d743`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:07.872640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6c39aa9c-1828-4e58-b275-9a301d0f9cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:07.967753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1273/2534] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:07.984116 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d1a0f-5e49-4faa-a620-1475946389ef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:07.986100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fd9d1a0f-5e49-4faa-a620-1475946389ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:08.057540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d1a0f-5e49-4faa-a620-1475946389ef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:08.061588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e1116-c55d-43d1-a4a5-568876a12d33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:08.062263 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e1116-c55d-43d1-a4a5-568876a12d33 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:08.062553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e1116-c55d-43d1-a4a5-568876a12d33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:08.063399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e1116-c55d-43d1-a4a5-568876a12d33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:08.132375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e1116-c55d-43d1-a4a5-568876a12d33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:08.134570 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e1116-c55d-43d1-a4a5-568876a12d33 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:08.147993 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1262/2535] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:17:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:17:08.231466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37958d3d-c1ac-4b8f-95d0-39625cfec42e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:08.232021 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37958d3d-c1ac-4b8f-95d0-39625cfec42e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:08.232726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37958d3d-c1ac-4b8f-95d0-39625cfec42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:08.240059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:08.255771 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-37958d3d-c1ac-4b8f-95d0-39625cfec42e None None] MFA Rules not processed for user `7f46019e78cc4e89bf877a4f5a7a8fe9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:08.257931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-37958d3d-c1ac-4b8f-95d0-39625cfec42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:08.343593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1274/2536] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:09.742790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e1116-c55d-43d1-a4a5-568876a12d33 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:09.745049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3e9e1116-c55d-43d1-a4a5-568876a12d33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:09.762673 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e1116-c55d-43d1-a4a5-568876a12d33 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:09.767358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bccc69-17f2-4e11-ba73-983929c5ba2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:09.768080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bccc69-17f2-4e11-ba73-983929c5ba2f service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:09.768362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bccc69-17f2-4e11-ba73-983929c5ba2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:09.768633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bccc69-17f2-4e11-ba73-983929c5ba2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:09.832342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bccc69-17f2-4e11-ba73-983929c5ba2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:09.834051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bccc69-17f2-4e11-ba73-983929c5ba2f service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:09.844992 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1263/2537] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:17:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:17:10.467604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37958d3d-c1ac-4b8f-95d0-39625cfec42e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:10.477967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37958d3d-c1ac-4b8f-95d0-39625cfec42e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:10.482348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3239f-46c9-4cba-bd19-b650d24a56e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:10.482954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3239f-46c9-4cba-bd19-b650d24a56e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:10.483228 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3239f-46c9-4cba-bd19-b650d24a56e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:10.483549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3239f-46c9-4cba-bd19-b650d24a56e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:10.492804 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3239f-46c9-4cba-bd19-b650d24a56e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:10.495309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3239f-46c9-4cba-bd19-b650d24a56e9 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:10.505568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1275/2538] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:17:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:17:14.502202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bccc69-17f2-4e11-ba73-983929c5ba2f service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:14.513299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:14.576275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bccc69-17f2-4e11-ba73-983929c5ba2f service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:14.581111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:14.581716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:14.582041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:14.582231 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] PATH_INFO: `/v3/users/a9cc2890a15f412b8adaee9e80953763` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:14.583297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9cc2890a15f412b8adaee9e80953763)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:14.586034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:14.644641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9cc2890a15f412b8adaee9e80953763'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:14.659318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9cc2890a15f412b8adaee9e80953763'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:14.660321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9cc2890a15f412b8adaee9e80953763'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:14.661279 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9cc2890a15f412b8adaee9e80953763'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:14.676085 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9cc2890a15f412b8adaee9e80953763'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:14.698321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1264/2539] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:17:14 2022] DELETE /identity/v3/users/a9cc2890a15f412b8adaee9e80953763 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:14.703111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571262e-5f21-44bd-bbb9-203c4b69303b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:14.703640 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571262e-5f21-44bd-bbb9-203c4b69303b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:14.704136 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571262e-5f21-44bd-bbb9-203c4b69303b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:14.732689 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2571262e-5f21-44bd-bbb9-203c4b69303b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:14.734250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2571262e-5f21-44bd-bbb9-203c4b69303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:14.810442 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1276/2540] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:14.820683 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:14.822460 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:14.894571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ca837-1d65-4342-992f-0896875e87e6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:14.898772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb7125-f4ed-4deb-a58a-0e655aecf75b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:14.899210 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb7125-f4ed-4deb-a58a-0e655aecf75b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:14.899861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb7125-f4ed-4deb-a58a-0e655aecf75b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:14.900142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb7125-f4ed-4deb-a58a-0e655aecf75b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:14.913436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb7125-f4ed-4deb-a58a-0e655aecf75b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:14.917427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb7125-f4ed-4deb-a58a-0e655aecf75b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:14.939225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1265/2541] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:17:15.509835 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571262e-5f21-44bd-bbb9-203c4b69303b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:15.511276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2571262e-5f21-44bd-bbb9-203c4b69303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:15.527139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571262e-5f21-44bd-bbb9-203c4b69303b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:15.532000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:15.532639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:15.532912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:15.533181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] PATH_INFO: `/v3/projects/07be24332ca94de3ae76838813b3b102` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:15.534366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=07be24332ca94de3ae76838813b3b102)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:15.536292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:15.595245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07be24332ca94de3ae76838813b3b102', 'request_id': 'req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:15.602762 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07be24332ca94de3ae76838813b3b102', 'request_id': 'req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:15.609401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07be24332ca94de3ae76838813b3b102', 'request_id': 'req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:15.621548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] The token cache is being invalidate because project 07be24332ca94de3ae76838813b3b102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:17:15.621853 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:15.622525 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:15.624037 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1277/2542] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:17:15 2022] DELETE /identity/v3/projects/07be24332ca94de3ae76838813b3b102 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:15.628092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb7125-f4ed-4deb-a58a-0e655aecf75b service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:15.638805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb7125-f4ed-4deb-a58a-0e655aecf75b service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:15.672848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:15.707534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:15.708300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:15.708647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:15.709010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] PATH_INFO: `/v3/users/9cecfea5d5a5453fb2757fc3adce9835` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:15.710272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cecfea5d5a5453fb2757fc3adce9835)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:15.712821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:15.767572 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cecfea5d5a5453fb2757fc3adce9835'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:15.778468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cecfea5d5a5453fb2757fc3adce9835'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:15.779266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cecfea5d5a5453fb2757fc3adce9835'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:15.780142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cecfea5d5a5453fb2757fc3adce9835'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:15.793691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cecfea5d5a5453fb2757fc3adce9835'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:15.829544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1266/2543] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:17:15 2022] DELETE /identity/v3/users/9cecfea5d5a5453fb2757fc3adce9835 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:16.157215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:16.158741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:16.175065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b4749-8c9e-42d8-a29d-729cada8a5fd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:16.213549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:16.280224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:16.280870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:16.281330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:16.281796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] PATH_INFO: `/v3/projects/4936ba803cd7455fa8cbe200034ee3a3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:16.283143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4936ba803cd7455fa8cbe200034ee3a3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:16.285575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:16.346576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4936ba803cd7455fa8cbe200034ee3a3', 'request_id': 'req-874540f0-c969-4ecd-99f6-7032110ff7f0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:16.357468 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4936ba803cd7455fa8cbe200034ee3a3', 'request_id': 'req-874540f0-c969-4ecd-99f6-7032110ff7f0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:16.364959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4936ba803cd7455fa8cbe200034ee3a3', 'request_id': 'req-874540f0-c969-4ecd-99f6-7032110ff7f0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:16.379182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] The token cache is being invalidate because project 4936ba803cd7455fa8cbe200034ee3a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:17:16.379469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:16.380318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:16.382167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1278/2544] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:17:16 2022] DELETE /identity/v3/projects/4936ba803cd7455fa8cbe200034ee3a3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:16.386984 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:16.388926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:16.406935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29f7b45-5b71-41ce-ab31-bbb0ecfcbe20 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:16.503027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:16.503663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:16.503968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:16.504237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] PATH_INFO: `/v3/users/7b1233c400554fc4b80a8c76f491d743` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:16.505374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b1233c400554fc4b80a8c76f491d743)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:16.507341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:16.558360 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b1233c400554fc4b80a8c76f491d743'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:16.563505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:16.569170 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b1233c400554fc4b80a8c76f491d743'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:16.569862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b1233c400554fc4b80a8c76f491d743'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:16.571859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b1233c400554fc4b80a8c76f491d743'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:16.579851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-44b097e8-5095-4b31-8869-84a19f960226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b1233c400554fc4b80a8c76f491d743'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:16.600539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1267/2545] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:17:16 2022] DELETE /identity/v3/users/7b1233c400554fc4b80a8c76f491d743 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:16.927055 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:16.928614 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:16.942383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874540f0-c969-4ecd-99f6-7032110ff7f0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:17.012700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e044df-f696-405f-8136-03c931641133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:17.013604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e044df-f696-405f-8136-03c931641133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.013921 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e044df-f696-405f-8136-03c931641133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.014204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e044df-f696-405f-8136-03c931641133 admin admin] PATH_INFO: `/v3/projects/33267721b8fb44fe877f680c8b609b7e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.015362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e044df-f696-405f-8136-03c931641133 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33267721b8fb44fe877f680c8b609b7e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:17.017307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e044df-f696-405f-8136-03c931641133 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:17.066756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-76e044df-f696-405f-8136-03c931641133 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33267721b8fb44fe877f680c8b609b7e', 'request_id': 'req-76e044df-f696-405f-8136-03c931641133'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:17.068500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:17.075692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-76e044df-f696-405f-8136-03c931641133 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33267721b8fb44fe877f680c8b609b7e', 'request_id': 'req-76e044df-f696-405f-8136-03c931641133'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:17.081865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-76e044df-f696-405f-8136-03c931641133 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33267721b8fb44fe877f680c8b609b7e', 'request_id': 'req-76e044df-f696-405f-8136-03c931641133'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:17.096396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-76e044df-f696-405f-8136-03c931641133 admin admin] The token cache is being invalidate because project 33267721b8fb44fe877f680c8b609b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:17:17.097232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-76e044df-f696-405f-8136-03c931641133 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:17.097344 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-76e044df-f696-405f-8136-03c931641133 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:17.098874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1279/2546] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:17:16 2022] DELETE /identity/v3/projects/33267721b8fb44fe877f680c8b609b7e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:17.104820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14710b0-26e3-42dc-b249-702db98cf056 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.105311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14710b0-26e3-42dc-b249-702db98cf056 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.105410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14710b0-26e3-42dc-b249-702db98cf056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.126377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e14710b0-26e3-42dc-b249-702db98cf056 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:17.127595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e14710b0-26e3-42dc-b249-702db98cf056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:17.191935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1268/2547] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:17.201458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbb00f-8956-4385-9bf6-782d69893f36 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.202035 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbb00f-8956-4385-9bf6-782d69893f36 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.202035 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadbb00f-8956-4385-9bf6-782d69893f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.224085 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fadbb00f-8956-4385-9bf6-782d69893f36 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:17.225607 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fadbb00f-8956-4385-9bf6-782d69893f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:17.237481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1280/2548] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:17.241924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14710b0-26e3-42dc-b249-702db98cf056 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:17.252999 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14710b0-26e3-42dc-b249-702db98cf056 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:17.257539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cb766-7824-4bb4-8a40-102f0dfb48ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:17.258183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cb766-7824-4bb4-8a40-102f0dfb48ea admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.258461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cb766-7824-4bb4-8a40-102f0dfb48ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.258811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cb766-7824-4bb4-8a40-102f0dfb48ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.259705 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cb766-7824-4bb4-8a40-102f0dfb48ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:17:17.260007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cb766-7824-4bb4-8a40-102f0dfb48ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:17.261590 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cb766-7824-4bb4-8a40-102f0dfb48ea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:17.273323 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1269/2549] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:17:17.279572 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbb00f-8956-4385-9bf6-782d69893f36 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:17.289005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadbb00f-8956-4385-9bf6-782d69893f36 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:17.294705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db89da-2956-4ccb-9c47-4729b09c9fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:17.295309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db89da-2956-4ccb-9c47-4729b09c9fde admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.295609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db89da-2956-4ccb-9c47-4729b09c9fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.295880 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db89da-2956-4ccb-9c47-4729b09c9fde admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.296499 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db89da-2956-4ccb-9c47-4729b09c9fde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:17.298461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db89da-2956-4ccb-9c47-4729b09c9fde admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:17.326335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1281/2550] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:17:17 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:17.330660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cb766-7824-4bb4-8a40-102f0dfb48ea admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:17.340644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cb766-7824-4bb4-8a40-102f0dfb48ea admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:17.399721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4ec8f-fd39-4493-9f61-580c147d249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:17.400434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4ec8f-fd39-4493-9f61-580c147d249f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.400751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4ec8f-fd39-4493-9f61-580c147d249f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.401026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4ec8f-fd39-4493-9f61-580c147d249f admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.401798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4ec8f-fd39-4493-9f61-580c147d249f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:17.403954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4ec8f-fd39-4493-9f61-580c147d249f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:17.427466 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1270/2551] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:17:17 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:17:17.431988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db89da-2956-4ccb-9c47-4729b09c9fde admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:17.442403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db89da-2956-4ccb-9c47-4729b09c9fde admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:17.447539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe1fdc-d429-47b2-9099-26eb4610f597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:17.448148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe1fdc-d429-47b2-9099-26eb4610f597 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.448429 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe1fdc-d429-47b2-9099-26eb4610f597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.448702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fe1fdc-d429-47b2-9099-26eb4610f597 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.449353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe1fdc-d429-47b2-9099-26eb4610f597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:17.451044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fe1fdc-d429-47b2-9099-26eb4610f597 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:17.461484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1282/2552] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:17.466379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4ec8f-fd39-4493-9f61-580c147d249f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:17.476921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4ec8f-fd39-4493-9f61-580c147d249f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:17.482469 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384e12c-fe9f-472c-ad7d-e3ae06954bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:17.483075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384e12c-fe9f-472c-ad7d-e3ae06954bce admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.483353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384e12c-fe9f-472c-ad7d-e3ae06954bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.483626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384e12c-fe9f-472c-ad7d-e3ae06954bce admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.484272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384e12c-fe9f-472c-ad7d-e3ae06954bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:17.485941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384e12c-fe9f-472c-ad7d-e3ae06954bce admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:17.502307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1271/2553] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:17:17.506666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe1fdc-d429-47b2-9099-26eb4610f597 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:17.518313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fe1fdc-d429-47b2-9099-26eb4610f597 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:17.523065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757723f-e946-44e5-97b9-c9034240806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:17.523734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757723f-e946-44e5-97b9-c9034240806e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.524003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757723f-e946-44e5-97b9-c9034240806e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.524784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757723f-e946-44e5-97b9-c9034240806e admin admin] PATH_INFO: `/v3/projects/c02c03286e7b4407bba15e05283f237f/users/b00b4b5e21fa43338bfc5557cd28912e/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.541426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757723f-e946-44e5-97b9-c9034240806e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02c03286e7b4407bba15e05283f237f, user_id=b00b4b5e21fa43338bfc5557cd28912e, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:17.544194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757723f-e946-44e5-97b9-c9034240806e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:17.559132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1283/2554] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:17:17 2022] PUT /identity/v3/projects/c02c03286e7b4407bba15e05283f237f/users/b00b4b5e21fa43338bfc5557cd28912e/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:17.569376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d726ff-e6ac-4379-a3bd-eee0a83d0578 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.569665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d726ff-e6ac-4379-a3bd-eee0a83d0578 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.569973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d726ff-e6ac-4379-a3bd-eee0a83d0578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.598776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-60d726ff-e6ac-4379-a3bd-eee0a83d0578 None None] MFA Rules not processed for user `b00b4b5e21fa43338bfc5557cd28912e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:17.600015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-60d726ff-e6ac-4379-a3bd-eee0a83d0578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:17.627140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:17.674872 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1272/2555] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:17.687415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757723f-e946-44e5-97b9-c9034240806e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:17.689491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b757723f-e946-44e5-97b9-c9034240806e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:17.755466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757723f-e946-44e5-97b9-c9034240806e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:17.759680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3e4aa-6f9f-4d53-98ae-f7c376e594ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:17.760282 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3e4aa-6f9f-4d53-98ae-f7c376e594ee service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.760557 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3e4aa-6f9f-4d53-98ae-f7c376e594ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.760826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3e4aa-6f9f-4d53-98ae-f7c376e594ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.769481 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d3e4aa-6f9f-4d53-98ae-f7c376e594ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:17.770933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d3e4aa-6f9f-4d53-98ae-f7c376e594ee service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:17.779691 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1284/2556] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:17:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:17:17.807898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d726ff-e6ac-4379-a3bd-eee0a83d0578 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:17.879662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d726ff-e6ac-4379-a3bd-eee0a83d0578 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:17.884503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:17.884976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.885689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.885689 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] PATH_INFO: `/v3/users/b00b4b5e21fa43338bfc5557cd28912e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.886857 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b00b4b5e21fa43338bfc5557cd28912e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:17.889028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:17.935046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b00b4b5e21fa43338bfc5557cd28912e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:17.942331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b00b4b5e21fa43338bfc5557cd28912e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:17.942950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b00b4b5e21fa43338bfc5557cd28912e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:17.943560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b00b4b5e21fa43338bfc5557cd28912e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:17.951345 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b00b4b5e21fa43338bfc5557cd28912e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:17.968900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1273/2557] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:17:17 2022] DELETE /identity/v3/users/b00b4b5e21fa43338bfc5557cd28912e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:17.973611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ffc3f-3841-4e48-ad78-78c8b19a4cf8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:17.973935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ffc3f-3841-4e48-ad78-78c8b19a4cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:17.974209 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ffc3f-3841-4e48-ad78-78c8b19a4cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:17.994901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8b1ffc3f-3841-4e48-ad78-78c8b19a4cf8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:17.996120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8b1ffc3f-3841-4e48-ad78-78c8b19a4cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:18.056253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1285/2558] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:17:18.065858 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:18.067272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:18.115397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:18.128007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d3f2c-e4fa-4055-bd8b-032f486c0ddc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:18.131868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29faf91-fe56-4657-8573-4d611cf5ff51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:18.132467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29faf91-fe56-4657-8573-4d611cf5ff51 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:18.132742 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29faf91-fe56-4657-8573-4d611cf5ff51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:18.133014 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29faf91-fe56-4657-8573-4d611cf5ff51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:18.142643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29faf91-fe56-4657-8573-4d611cf5ff51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:18.144064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29faf91-fe56-4657-8573-4d611cf5ff51 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:18.154423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1274/2559] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:17:18.443566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ffc3f-3841-4e48-ad78-78c8b19a4cf8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:18.447635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8b1ffc3f-3841-4e48-ad78-78c8b19a4cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:18.471193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ffc3f-3841-4e48-ad78-78c8b19a4cf8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:18.478045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:18.478892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:18.479180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:18.479455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] PATH_INFO: `/v3/projects/c02c03286e7b4407bba15e05283f237f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:18.480674 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c02c03286e7b4407bba15e05283f237f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:18.482781 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:18.496551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:18.542191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c02c03286e7b4407bba15e05283f237f', 'request_id': 'req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:18.553489 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c02c03286e7b4407bba15e05283f237f', 'request_id': 'req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:18.561563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c02c03286e7b4407bba15e05283f237f', 'request_id': 'req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:18.574939 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] The token cache is being invalidate because project c02c03286e7b4407bba15e05283f237f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:17:18.575223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:18.575901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:17:18.577427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1286/2560] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:17:18 2022] DELETE /identity/v3/projects/c02c03286e7b4407bba15e05283f237f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:17:28.149310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:28.570872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:37.330486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9e030-e4a1-42ff-befe-39007dd9fc16 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:37.330946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9e030-e4a1-42ff-befe-39007dd9fc16 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:37.331040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9e030-e4a1-42ff-befe-39007dd9fc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:37.361087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-01c9e030-e4a1-42ff-befe-39007dd9fc16 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:37.362347 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-01c9e030-e4a1-42ff-befe-39007dd9fc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:37.427870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1275/2561] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:17:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:37.476506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:37.478508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:37.552621 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fefb45a-38ab-419c-a2f1-10b8e7fddd01 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:37.558406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998bbbb-93af-4be3-8dc0-b584490b3c35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:37.559078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998bbbb-93af-4be3-8dc0-b584490b3c35 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:37.559391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998bbbb-93af-4be3-8dc0-b584490b3c35 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:37.559690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998bbbb-93af-4be3-8dc0-b584490b3c35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:37.568934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998bbbb-93af-4be3-8dc0-b584490b3c35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:37.570663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998bbbb-93af-4be3-8dc0-b584490b3c35 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:37.579141 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1287/2562] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:17:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:17:47.421397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=32 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:47.575847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=32 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:17:54.285480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f8820-bd2a-4180-8b2d-9eeb1e185fa4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:54.286112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f8820-bd2a-4180-8b2d-9eeb1e185fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:54.286112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f8820-bd2a-4180-8b2d-9eeb1e185fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:54.307215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fe8f8820-bd2a-4180-8b2d-9eeb1e185fa4 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:17:54.308548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe8f8820-bd2a-4180-8b2d-9eeb1e185fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:17:54.316326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1276/2563] 10.222.0.16 () {58 vars in 1009 bytes} [Fri Aug 19 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:17:54.328286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998bbbb-93af-4be3-8dc0-b584490b3c35 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:17:54.339668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998bbbb-93af-4be3-8dc0-b584490b3c35 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:17:54.344917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874b6a5-f1a2-40f0-a36d-738125b2fd93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:17:54.345534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874b6a5-f1a2-40f0-a36d-738125b2fd93 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:17:54.345847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874b6a5-f1a2-40f0-a36d-738125b2fd93 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:17:54.346127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874b6a5-f1a2-40f0-a36d-738125b2fd93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:17:54.355636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874b6a5-f1a2-40f0-a36d-738125b2fd93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:17:54.357246 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874b6a5-f1a2-40f0-a36d-738125b2fd93 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:17:54.367773 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1288/2564] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:17:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:18:04.302095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:04.363737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:06.991022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f8820-bd2a-4180-8b2d-9eeb1e185fa4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:06.992430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fe8f8820-bd2a-4180-8b2d-9eeb1e185fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:07.050676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f8820-bd2a-4180-8b2d-9eeb1e185fa4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:07.054297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f204a8-49f2-42dd-9016-e79e3fb6748a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:07.054902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f204a8-49f2-42dd-9016-e79e3fb6748a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:07.055183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f204a8-49f2-42dd-9016-e79e3fb6748a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:07.055456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f204a8-49f2-42dd-9016-e79e3fb6748a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:07.057387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-75f204a8-49f2-42dd-9016-e79e3fb6748a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:07.152636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f204a8-49f2-42dd-9016-e79e3fb6748a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:07.154133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f204a8-49f2-42dd-9016-e79e3fb6748a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:07.175438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1277/2565] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:18:08.310150 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874b6a5-f1a2-40f0-a36d-738125b2fd93 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:08.319782 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874b6a5-f1a2-40f0-a36d-738125b2fd93 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:08.323821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ac0f2-46f7-40e4-b9d3-7ca0604958de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:08.324418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ac0f2-46f7-40e4-b9d3-7ca0604958de service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:08.324692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ac0f2-46f7-40e4-b9d3-7ca0604958de service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:08.324960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ac0f2-46f7-40e4-b9d3-7ca0604958de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:08.326702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b25ac0f2-46f7-40e4-b9d3-7ca0604958de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:08.341459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ac0f2-46f7-40e4-b9d3-7ca0604958de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:08.342892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ac0f2-46f7-40e4-b9d3-7ca0604958de service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:08.355394 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1289/2566] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:18:16.336787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d639909-c097-4768-a945-e0aee0c0d6f3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:16.338014 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d639909-c097-4768-a945-e0aee0c0d6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:16.338014 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d639909-c097-4768-a945-e0aee0c0d6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:16.356632 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6d639909-c097-4768-a945-e0aee0c0d6f3 None None] MFA Rules not processed for user `bdce5268fa7c466280dddc19869dce87`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:16.357800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6d639909-c097-4768-a945-e0aee0c0d6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:16.364831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1278/2567] 10.222.0.16 () {58 vars in 967 bytes} [Fri Aug 19 21:18:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:16.377580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ac0f2-46f7-40e4-b9d3-7ca0604958de service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:16.389105 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ac0f2-46f7-40e4-b9d3-7ca0604958de service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:16.395841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799e67b-7fe6-4438-8755-904f86c9c5a8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a0906b8b134c178b4dece2891f1596', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:16.396449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799e67b-7fe6-4438-8755-904f86c9c5a8 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:16.396726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799e67b-7fe6-4438-8755-904f86c9c5a8 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:16.396998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799e67b-7fe6-4438-8755-904f86c9c5a8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:16.406004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799e67b-7fe6-4438-8755-904f86c9c5a8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:16.407585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799e67b-7fe6-4438-8755-904f86c9c5a8 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:16.416703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1290/2568] 10.222.0.16 () {60 vars in 1384 bytes} [Fri Aug 19 21:18:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 19 21:18:26.351010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=66 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:26.412618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=12 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:31.908883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d639909-c097-4768-a945-e0aee0c0d6f3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:31.918636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d639909-c097-4768-a945-e0aee0c0d6f3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:31.922231 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3db85-cff2-4b62-b15f-9ba1a0145ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:31.922821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3db85-cff2-4b62-b15f-9ba1a0145ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:31.923094 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3db85-cff2-4b62-b15f-9ba1a0145ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:31.923362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3db85-cff2-4b62-b15f-9ba1a0145ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:31.925062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ce3db85-cff2-4b62-b15f-9ba1a0145ae2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:31.990375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3db85-cff2-4b62-b15f-9ba1a0145ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:31.991774 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3db85-cff2-4b62-b15f-9ba1a0145ae2 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:32.002167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1279/2569] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:18:35.189155 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799e67b-7fe6-4438-8755-904f86c9c5a8 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:35.190667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9799e67b-7fe6-4438-8755-904f86c9c5a8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:35.205096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799e67b-7fe6-4438-8755-904f86c9c5a8 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:35.209796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:35.210427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:35.210711 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:35.210983 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] PATH_INFO: `/v3/users/8810b3e211d448cea9a506d9183dc13d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:35.226062 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8810b3e211d448cea9a506d9183dc13d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:35.228523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:35.275999 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8810b3e211d448cea9a506d9183dc13d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:35.276661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8810b3e211d448cea9a506d9183dc13d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:35.277279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8810b3e211d448cea9a506d9183dc13d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:35.283875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8810b3e211d448cea9a506d9183dc13d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:35.296298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dd844e63-2866-433c-a611-451a8e8783f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8810b3e211d448cea9a506d9183dc13d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:35.309066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1291/2570] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:18:35 2022] DELETE /identity/v3/users/8810b3e211d448cea9a506d9183dc13d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:35.482043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3db85-cff2-4b62-b15f-9ba1a0145ae2 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:35.492434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3db85-cff2-4b62-b15f-9ba1a0145ae2 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:35.552018 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:35.552647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:35.552941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:35.553206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] PATH_INFO: `/v3/projects/def1895e8f3b40f1b608240c60a0066a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:35.564419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] RBAC: Authorizing `identity:delete_project(project_id=def1895e8f3b40f1b608240c60a0066a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:35.566285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:35.612483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def1895e8f3b40f1b608240c60a0066a', 'request_id': 'req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:35.619191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def1895e8f3b40f1b608240c60a0066a', 'request_id': 'req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:35.624815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def1895e8f3b40f1b608240c60a0066a', 'request_id': 'req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:35.638995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] The token cache is being invalidate because project def1895e8f3b40f1b608240c60a0066a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:18:35.639289 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:35.640274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb2f3735-91b9-4f7c-aa31-82a47adfd01b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:35.641787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1280/2571] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:18:35 2022] DELETE /identity/v3/projects/def1895e8f3b40f1b608240c60a0066a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:35.646670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540e86b-7804-4b68-b802-349b0bdfa891 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:35.647022 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540e86b-7804-4b68-b802-349b0bdfa891 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:35.647302 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540e86b-7804-4b68-b802-349b0bdfa891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:35.669296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7540e86b-7804-4b68-b802-349b0bdfa891 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:35.670537 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7540e86b-7804-4b68-b802-349b0bdfa891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:35.734494 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1292/2572] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:35.742440 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeca43a-4d33-4021-9dab-74dd2d869324 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:35.742730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeca43a-4d33-4021-9dab-74dd2d869324 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:35.743006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeca43a-4d33-4021-9dab-74dd2d869324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:35.765594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9eeca43a-4d33-4021-9dab-74dd2d869324 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:35.766997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9eeca43a-4d33-4021-9dab-74dd2d869324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:35.776699 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1281/2573] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:35.781048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540e86b-7804-4b68-b802-349b0bdfa891 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:35.792041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540e86b-7804-4b68-b802-349b0bdfa891 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:35.796810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f82ab4-4854-4d7b-b53d-69d1ffb308e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:35.797395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f82ab4-4854-4d7b-b53d-69d1ffb308e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:35.797737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f82ab4-4854-4d7b-b53d-69d1ffb308e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:35.797960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f82ab4-4854-4d7b-b53d-69d1ffb308e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:35.798613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f82ab4-4854-4d7b-b53d-69d1ffb308e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:18:35.798918 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f82ab4-4854-4d7b-b53d-69d1ffb308e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:35.800540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f82ab4-4854-4d7b-b53d-69d1ffb308e5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:35.813591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1293/2574] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:18:35.818099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeca43a-4d33-4021-9dab-74dd2d869324 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:35.827406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeca43a-4d33-4021-9dab-74dd2d869324 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:35.832075 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c481a9-3e95-4a98-bcec-9d4eabed9549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:35.832675 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c481a9-3e95-4a98-bcec-9d4eabed9549 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:35.832951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c481a9-3e95-4a98-bcec-9d4eabed9549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:35.833219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c481a9-3e95-4a98-bcec-9d4eabed9549 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:35.833885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c481a9-3e95-4a98-bcec-9d4eabed9549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:35.835778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c481a9-3e95-4a98-bcec-9d4eabed9549 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:35.841337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:35.874641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1282/2575] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:18:35 2022] POST /identity/v3/projects => generated 393 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:18:35.878683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f82ab4-4854-4d7b-b53d-69d1ffb308e5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:35.889025 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f82ab4-4854-4d7b-b53d-69d1ffb308e5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:35.934646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:35.947821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471a40a-c4b2-424a-aee9-6b21e66dc711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:35.948450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a40a-c4b2-424a-aee9-6b21e66dc711 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:35.948729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a40a-c4b2-424a-aee9-6b21e66dc711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:35.948995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8471a40a-c4b2-424a-aee9-6b21e66dc711 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:35.949624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471a40a-c4b2-424a-aee9-6b21e66dc711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:35.951573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8471a40a-c4b2-424a-aee9-6b21e66dc711 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:35.972325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1294/2576] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:18:35 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:18:35.976703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c481a9-3e95-4a98-bcec-9d4eabed9549 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:35.987750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c481a9-3e95-4a98-bcec-9d4eabed9549 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:35.992375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b86fd-e956-4b83-a407-c0023f3be9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:35.992972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b86fd-e956-4b83-a407-c0023f3be9d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:35.993304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b86fd-e956-4b83-a407-c0023f3be9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:35.994441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b86fd-e956-4b83-a407-c0023f3be9d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:35.994824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b86fd-e956-4b83-a407-c0023f3be9d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:35.996415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b86fd-e956-4b83-a407-c0023f3be9d7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.006199 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1283/2577] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:36.010523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471a40a-c4b2-424a-aee9-6b21e66dc711 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:36.020730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8471a40a-c4b2-424a-aee9-6b21e66dc711 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:36.025484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758abe1-cf29-432e-a972-69b2035ebadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:36.026124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758abe1-cf29-432e-a972-69b2035ebadf admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.026405 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758abe1-cf29-432e-a972-69b2035ebadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.026676 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758abe1-cf29-432e-a972-69b2035ebadf admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.027311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758abe1-cf29-432e-a972-69b2035ebadf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:36.028913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758abe1-cf29-432e-a972-69b2035ebadf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.039679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1295/2578] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:36.044387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b86fd-e956-4b83-a407-c0023f3be9d7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:36.054432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b86fd-e956-4b83-a407-c0023f3be9d7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:36.059966 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67763a3d-e35c-41c9-8c58-342ae7531042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:36.060514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763a3d-e35c-41c9-8c58-342ae7531042 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.060793 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763a3d-e35c-41c9-8c58-342ae7531042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.061071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763a3d-e35c-41c9-8c58-342ae7531042 admin admin] PATH_INFO: `/v3/projects/bfa770b874a4442ba2bfb20c8fb24b16/users/ab0885c0cc4b44e19ac8f1e7eb5b67b2/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.077431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67763a3d-e35c-41c9-8c58-342ae7531042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa770b874a4442ba2bfb20c8fb24b16, user_id=ab0885c0cc4b44e19ac8f1e7eb5b67b2, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:36.080072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67763a3d-e35c-41c9-8c58-342ae7531042 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.095597 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1284/2579] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:18:36 2022] PUT /identity/v3/projects/bfa770b874a4442ba2bfb20c8fb24b16/users/ab0885c0cc4b44e19ac8f1e7eb5b67b2/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:36.107696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2c79e-e290-47f4-8b96-c0ee5199af8d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.108009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2c79e-e290-47f4-8b96-c0ee5199af8d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.108259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2c79e-e290-47f4-8b96-c0ee5199af8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.136735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-06b2c79e-e290-47f4-8b96-c0ee5199af8d None None] MFA Rules not processed for user `ab0885c0cc4b44e19ac8f1e7eb5b67b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:36.137981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-06b2c79e-e290-47f4-8b96-c0ee5199af8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:36.204595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1296/2580] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:36.209538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67763a3d-e35c-41c9-8c58-342ae7531042 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:36.273944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67763a3d-e35c-41c9-8c58-342ae7531042 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:36.279136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90908b-e309-4752-a886-e1b6d6d16f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:36.279791 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90908b-e309-4752-a886-e1b6d6d16f8b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.280062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90908b-e309-4752-a886-e1b6d6d16f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.280329 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90908b-e309-4752-a886-e1b6d6d16f8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.280943 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90908b-e309-4752-a886-e1b6d6d16f8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:36.282868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90908b-e309-4752-a886-e1b6d6d16f8b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.311519 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1285/2581] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:18:36 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:18:36.315811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2c79e-e290-47f4-8b96-c0ee5199af8d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:36.327382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2c79e-e290-47f4-8b96-c0ee5199af8d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:36.384622 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d28bb-6eba-4c2b-98e0-dad2ce7100bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:36.385036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d28bb-6eba-4c2b-98e0-dad2ce7100bd admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.385317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d28bb-6eba-4c2b-98e0-dad2ce7100bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.385590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06d28bb-6eba-4c2b-98e0-dad2ce7100bd admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.386358 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d28bb-6eba-4c2b-98e0-dad2ce7100bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:36.388195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06d28bb-6eba-4c2b-98e0-dad2ce7100bd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.409248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1297/2582] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:18:36 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:18:36.413424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90908b-e309-4752-a886-e1b6d6d16f8b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:36.423704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90908b-e309-4752-a886-e1b6d6d16f8b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:36.428300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63f22a-a739-425e-bb2f-be819921c1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:36.428889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63f22a-a739-425e-bb2f-be819921c1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.429163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63f22a-a739-425e-bb2f-be819921c1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.429431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd63f22a-a739-425e-bb2f-be819921c1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.430105 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63f22a-a739-425e-bb2f-be819921c1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:36.432236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd63f22a-a739-425e-bb2f-be819921c1b7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.442778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1286/2583] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:36.449046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d28bb-6eba-4c2b-98e0-dad2ce7100bd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:36.460072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06d28bb-6eba-4c2b-98e0-dad2ce7100bd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:36.464930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e0272-cb1d-4751-b702-6ea5ffd3e015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:36.465611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e0272-cb1d-4751-b702-6ea5ffd3e015 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.465927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e0272-cb1d-4751-b702-6ea5ffd3e015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.466207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e0272-cb1d-4751-b702-6ea5ffd3e015 admin admin] PATH_INFO: `/v3/projects/56798a2cf0b848b394ceff085e8cd5fa/users/39e4f1ed6daa41f4b8781273b68d90af/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.481595 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e0272-cb1d-4751-b702-6ea5ffd3e015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56798a2cf0b848b394ceff085e8cd5fa, user_id=39e4f1ed6daa41f4b8781273b68d90af, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:36.484315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465e0272-cb1d-4751-b702-6ea5ffd3e015 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.497804 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1298/2584] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:18:36 2022] PUT /identity/v3/projects/56798a2cf0b848b394ceff085e8cd5fa/users/39e4f1ed6daa41f4b8781273b68d90af/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:36.501838 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63f22a-a739-425e-bb2f-be819921c1b7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:36.512298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd63f22a-a739-425e-bb2f-be819921c1b7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:36.567880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697bba90-f0d2-40b0-b538-88abf1b02a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:36.568723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bba90-f0d2-40b0-b538-88abf1b02a8d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.569121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bba90-f0d2-40b0-b538-88abf1b02a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.569405 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697bba90-f0d2-40b0-b538-88abf1b02a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.570096 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bba90-f0d2-40b0-b538-88abf1b02a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:36.572216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697bba90-f0d2-40b0-b538-88abf1b02a8d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.582083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1287/2585] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:36.586426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e0272-cb1d-4751-b702-6ea5ffd3e015 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:36.597744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465e0272-cb1d-4751-b702-6ea5ffd3e015 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:36.602325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaba3b4-7892-4062-93be-6649aba7f9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:36.602975 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaba3b4-7892-4062-93be-6649aba7f9c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.603258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaba3b4-7892-4062-93be-6649aba7f9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.603530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaba3b4-7892-4062-93be-6649aba7f9c1 admin admin] PATH_INFO: `/v3/projects/56798a2cf0b848b394ceff085e8cd5fa/users/39e4f1ed6daa41f4b8781273b68d90af/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.605437 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaba3b4-7892-4062-93be-6649aba7f9c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56798a2cf0b848b394ceff085e8cd5fa, user_id=39e4f1ed6daa41f4b8781273b68d90af, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:36.608133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaba3b4-7892-4062-93be-6649aba7f9c1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.618433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1299/2586] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:18:36 2022] PUT /identity/v3/projects/56798a2cf0b848b394ceff085e8cd5fa/users/39e4f1ed6daa41f4b8781273b68d90af/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:36.628800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3dc3f-2d36-4838-b8fe-0f0d5878366e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.629088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3dc3f-2d36-4838-b8fe-0f0d5878366e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.629366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3dc3f-2d36-4838-b8fe-0f0d5878366e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.657133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:36.660647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-efc3dc3f-2d36-4838-b8fe-0f0d5878366e None None] MFA Rules not processed for user `39e4f1ed6daa41f4b8781273b68d90af`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:36.662746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-efc3dc3f-2d36-4838-b8fe-0f0d5878366e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:36.735739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1288/2587] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:36.747619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaba3b4-7892-4062-93be-6649aba7f9c1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:36.749310 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ebaba3b4-7892-4062-93be-6649aba7f9c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:36.760336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:36.811707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaba3b4-7892-4062-93be-6649aba7f9c1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:36.815920 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e422ba-26d5-4a54-82ea-2f97ea1ac7fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:36.816742 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e422ba-26d5-4a54-82ea-2f97ea1ac7fd service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.817025 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e422ba-26d5-4a54-82ea-2f97ea1ac7fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.817325 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e422ba-26d5-4a54-82ea-2f97ea1ac7fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.827950 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e422ba-26d5-4a54-82ea-2f97ea1ac7fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:36.829391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e422ba-26d5-4a54-82ea-2f97ea1ac7fd service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.839654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1300/2588] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:36 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:18:36.886994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3dc3f-2d36-4838-b8fe-0f0d5878366e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:36.888456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-efc3dc3f-2d36-4838-b8fe-0f0d5878366e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:36.901402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3dc3f-2d36-4838-b8fe-0f0d5878366e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:36.905068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89241c24-5635-4867-9900-15acd8246a77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:36.905662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89241c24-5635-4867-9900-15acd8246a77 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:36.905986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89241c24-5635-4867-9900-15acd8246a77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:36.906252 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89241c24-5635-4867-9900-15acd8246a77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:36.966834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89241c24-5635-4867-9900-15acd8246a77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:36.968201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89241c24-5635-4867-9900-15acd8246a77 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:36.977144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1289/2589] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:18:40.265925 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e422ba-26d5-4a54-82ea-2f97ea1ac7fd service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:40.328564 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e422ba-26d5-4a54-82ea-2f97ea1ac7fd service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:40.333172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:40.333830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:40.334116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:40.334395 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] PATH_INFO: `/v3/users/ab0885c0cc4b44e19ac8f1e7eb5b67b2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:40.335587 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0885c0cc4b44e19ac8f1e7eb5b67b2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:40.338167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:40.385821 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0885c0cc4b44e19ac8f1e7eb5b67b2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:40.386417 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0885c0cc4b44e19ac8f1e7eb5b67b2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:40.387028 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0885c0cc4b44e19ac8f1e7eb5b67b2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:40.394796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0885c0cc4b44e19ac8f1e7eb5b67b2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:40.407581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0885c0cc4b44e19ac8f1e7eb5b67b2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:40.419812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1301/2590] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:18:40 2022] DELETE /identity/v3/users/ab0885c0cc4b44e19ac8f1e7eb5b67b2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:40.424281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761d750-e366-415e-9d0e-578ca2ff022b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:40.424739 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761d750-e366-415e-9d0e-578ca2ff022b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:40.424851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761d750-e366-415e-9d0e-578ca2ff022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:40.443416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d761d750-e366-415e-9d0e-578ca2ff022b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:40.444624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d761d750-e366-415e-9d0e-578ca2ff022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:40.504849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1290/2591] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:40.515391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:40.516832 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:40.551642 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:40.578599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e49bc0-e47a-4475-9ffe-d395def6d835 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:40.582191 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12919f-ebc3-4fbe-a1ce-13c46e4fb0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:40.582795 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12919f-ebc3-4fbe-a1ce-13c46e4fb0aa service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:40.583069 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12919f-ebc3-4fbe-a1ce-13c46e4fb0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:40.583335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12919f-ebc3-4fbe-a1ce-13c46e4fb0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:40.593039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12919f-ebc3-4fbe-a1ce-13c46e4fb0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:40.594490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12919f-ebc3-4fbe-a1ce-13c46e4fb0aa service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:40.605106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1302/2592] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:18:40.881056 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761d750-e366-415e-9d0e-578ca2ff022b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:40.882508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d761d750-e366-415e-9d0e-578ca2ff022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:40.895897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761d750-e366-415e-9d0e-578ca2ff022b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:40.901526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:40.902196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:40.902471 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:40.902743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] PATH_INFO: `/v3/projects/bfa770b874a4442ba2bfb20c8fb24b16` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:40.903868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa770b874a4442ba2bfb20c8fb24b16)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:40.905787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:40.941086 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:40.955525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa770b874a4442ba2bfb20c8fb24b16', 'request_id': 'req-ca856af6-9a68-4679-956c-764131ad4525'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:40.961911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa770b874a4442ba2bfb20c8fb24b16', 'request_id': 'req-ca856af6-9a68-4679-956c-764131ad4525'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:40.968955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa770b874a4442ba2bfb20c8fb24b16', 'request_id': 'req-ca856af6-9a68-4679-956c-764131ad4525'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:40.982543 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] The token cache is being invalidate because project bfa770b874a4442ba2bfb20c8fb24b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:18:40.982831 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:40.983461 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:40.984944 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1291/2593] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:18:40 2022] DELETE /identity/v3/projects/bfa770b874a4442ba2bfb20c8fb24b16 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:40.989168 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12919f-ebc3-4fbe-a1ce-13c46e4fb0aa service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:40.999107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12919f-ebc3-4fbe-a1ce-13c46e4fb0aa service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:41.063790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:41.064517 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:41.064798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:41.065070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] PATH_INFO: `/v3/users/39e4f1ed6daa41f4b8781273b68d90af` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:41.066229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39e4f1ed6daa41f4b8781273b68d90af)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:41.068143 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:41.117268 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39e4f1ed6daa41f4b8781273b68d90af'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:41.117954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39e4f1ed6daa41f4b8781273b68d90af'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:41.118627 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39e4f1ed6daa41f4b8781273b68d90af'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:41.126336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39e4f1ed6daa41f4b8781273b68d90af'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:41.138946 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f67899b8-10f3-433e-8ee7-a47491576037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39e4f1ed6daa41f4b8781273b68d90af'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:41.151910 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1303/2594] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:18:40 2022] DELETE /identity/v3/users/39e4f1ed6daa41f4b8781273b68d90af => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:41.417631 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:41.419219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:41.432764 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856af6-9a68-4679-956c-764131ad4525 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:41.490882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:41.491511 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:41.491789 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:41.492061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] PATH_INFO: `/v3/projects/56798a2cf0b848b394ceff085e8cd5fa` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:41.493176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56798a2cf0b848b394ceff085e8cd5fa)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:41.495756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:41.542221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56798a2cf0b848b394ceff085e8cd5fa', 'request_id': 'req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:41.548290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56798a2cf0b848b394ceff085e8cd5fa', 'request_id': 'req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:41.553797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56798a2cf0b848b394ceff085e8cd5fa', 'request_id': 'req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:41.566566 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] The token cache is being invalidate because project 56798a2cf0b848b394ceff085e8cd5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:18:41.566856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:41.567487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f1ec5ead-c50a-4b52-bdd6-0cba3b73a240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:41.568938 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1292/2595] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:18:41 2022] DELETE /identity/v3/projects/56798a2cf0b848b394ceff085e8cd5fa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:41.574663 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27626-3891-4c65-995a-60ecfddde3f3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:41.575156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27626-3891-4c65-995a-60ecfddde3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:41.575270 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d27626-3891-4c65-995a-60ecfddde3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:41.594600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-61d27626-3891-4c65-995a-60ecfddde3f3 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:41.595805 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-61d27626-3891-4c65-995a-60ecfddde3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:41.652555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:41.668155 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1304/2596] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:41.675833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b0d48-7236-4aa4-9202-3910bc99b487 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:41.676122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b0d48-7236-4aa4-9202-3910bc99b487 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:41.676398 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b0d48-7236-4aa4-9202-3910bc99b487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:41.698516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-086b0d48-7236-4aa4-9202-3910bc99b487 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:41.699749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-086b0d48-7236-4aa4-9202-3910bc99b487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:41.707558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1293/2597] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:41.711813 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27626-3891-4c65-995a-60ecfddde3f3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:41.721743 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d27626-3891-4c65-995a-60ecfddde3f3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:41.726355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a12dd-3794-455a-8315-4ae2b6afb206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:41.726962 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a12dd-3794-455a-8315-4ae2b6afb206 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:41.727240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a12dd-3794-455a-8315-4ae2b6afb206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:41.727512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22a12dd-3794-455a-8315-4ae2b6afb206 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:41.728704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a12dd-3794-455a-8315-4ae2b6afb206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:18:41.729239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a12dd-3794-455a-8315-4ae2b6afb206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:41.730633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22a12dd-3794-455a-8315-4ae2b6afb206 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:41.742817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1305/2598] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:18:41.747991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b0d48-7236-4aa4-9202-3910bc99b487 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:41.757371 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b0d48-7236-4aa4-9202-3910bc99b487 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:41.762445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4bd4d-d8d4-4cc1-a7bd-3e690b4b573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:41.763051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4bd4d-d8d4-4cc1-a7bd-3e690b4b573d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:41.763331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4bd4d-d8d4-4cc1-a7bd-3e690b4b573d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:41.763607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4bd4d-d8d4-4cc1-a7bd-3e690b4b573d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:41.764244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4bd4d-d8d4-4cc1-a7bd-3e690b4b573d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:41.766512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4bd4d-d8d4-4cc1-a7bd-3e690b4b573d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:41.793342 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1294/2599] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:18:41 2022] POST /identity/v3/projects => generated 409 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:18:41.797656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a12dd-3794-455a-8315-4ae2b6afb206 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:41.808986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22a12dd-3794-455a-8315-4ae2b6afb206 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:41.865690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d5c97-b805-4696-badd-536cd0c40a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:41.866328 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d5c97-b805-4696-badd-536cd0c40a2f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:41.866603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d5c97-b805-4696-badd-536cd0c40a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:41.866916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d5c97-b805-4696-badd-536cd0c40a2f admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:41.867532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d5c97-b805-4696-badd-536cd0c40a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:41.870323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d5c97-b805-4696-badd-536cd0c40a2f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:41.890958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1306/2600] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:18:41 2022] POST /identity/v3/users => generated 305 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:18:41.895264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4bd4d-d8d4-4cc1-a7bd-3e690b4b573d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:41.905828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4bd4d-d8d4-4cc1-a7bd-3e690b4b573d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:41.910335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b011b1-60c9-40a2-bb4e-5ac965cfa048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:41.910923 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b011b1-60c9-40a2-bb4e-5ac965cfa048 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:41.911198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b011b1-60c9-40a2-bb4e-5ac965cfa048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:41.911469 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b011b1-60c9-40a2-bb4e-5ac965cfa048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:41.912105 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b011b1-60c9-40a2-bb4e-5ac965cfa048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:41.913687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b011b1-60c9-40a2-bb4e-5ac965cfa048 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:41.924913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1295/2601] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:41.928856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d5c97-b805-4696-badd-536cd0c40a2f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:41.939519 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d5c97-b805-4696-badd-536cd0c40a2f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:41.944337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6b8d23-6bd9-4f52-be13-9ca562a260db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:41.944936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b8d23-6bd9-4f52-be13-9ca562a260db admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:41.945216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b8d23-6bd9-4f52-be13-9ca562a260db admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:41.945490 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6b8d23-6bd9-4f52-be13-9ca562a260db admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:41.946187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b8d23-6bd9-4f52-be13-9ca562a260db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:41.947876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6b8d23-6bd9-4f52-be13-9ca562a260db admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:41.960236 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1307/2602] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:41.964458 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b011b1-60c9-40a2-bb4e-5ac965cfa048 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:41.974890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b011b1-60c9-40a2-bb4e-5ac965cfa048 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:41.981221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d9e6e-5cef-4d98-93f8-7bd5342890ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:41.982038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d9e6e-5cef-4d98-93f8-7bd5342890ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:41.982428 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d9e6e-5cef-4d98-93f8-7bd5342890ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:41.982824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d9e6e-5cef-4d98-93f8-7bd5342890ae admin admin] PATH_INFO: `/v3/projects/bce8455f824f4e69ba1b4e0f38a4196a/users/e43a2a2e84bb40dc92797b664fbaea44/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:41.996307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:42.004538 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d9e6e-5cef-4d98-93f8-7bd5342890ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce8455f824f4e69ba1b4e0f38a4196a, user_id=e43a2a2e84bb40dc92797b664fbaea44, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:42.007668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d9e6e-5cef-4d98-93f8-7bd5342890ae admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:42.021004 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1296/2603] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:18:41 2022] PUT /identity/v3/projects/bce8455f824f4e69ba1b4e0f38a4196a/users/e43a2a2e84bb40dc92797b664fbaea44/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:18:42.033048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faad3a7-a9c3-4101-a243-ce3b9da2338e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:42.033526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faad3a7-a9c3-4101-a243-ce3b9da2338e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:42.033847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faad3a7-a9c3-4101-a243-ce3b9da2338e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:42.062626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5faad3a7-a9c3-4101-a243-ce3b9da2338e None None] MFA Rules not processed for user `e43a2a2e84bb40dc92797b664fbaea44`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:42.063902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5faad3a7-a9c3-4101-a243-ce3b9da2338e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:42.128751 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1308/2604] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:42.133311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d9e6e-5cef-4d98-93f8-7bd5342890ae admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:42.194107 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d9e6e-5cef-4d98-93f8-7bd5342890ae admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:42.198871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3024cc8-5964-4ea9-bd46-7822a8ec7203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:42.199465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3024cc8-5964-4ea9-bd46-7822a8ec7203 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:42.199743 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3024cc8-5964-4ea9-bd46-7822a8ec7203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:42.200016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3024cc8-5964-4ea9-bd46-7822a8ec7203 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:42.200646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3024cc8-5964-4ea9-bd46-7822a8ec7203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:42.202607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3024cc8-5964-4ea9-bd46-7822a8ec7203 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:42.229996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1297/2605] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:18:42 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:18:42.234197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faad3a7-a9c3-4101-a243-ce3b9da2338e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:42.245160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faad3a7-a9c3-4101-a243-ce3b9da2338e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:42.288755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:42.301567 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044d0ed-fbaa-4654-a57c-0317937b71ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:42.302203 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044d0ed-fbaa-4654-a57c-0317937b71ee admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:42.302486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044d0ed-fbaa-4654-a57c-0317937b71ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:42.302755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044d0ed-fbaa-4654-a57c-0317937b71ee admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:42.303385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044d0ed-fbaa-4654-a57c-0317937b71ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:42.305308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044d0ed-fbaa-4654-a57c-0317937b71ee admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:42.326101 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1309/2606] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:18:42 2022] POST /identity/v3/users => generated 311 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:18:42.330390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3024cc8-5964-4ea9-bd46-7822a8ec7203 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:42.341741 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3024cc8-5964-4ea9-bd46-7822a8ec7203 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:42.346621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a4c3c-fba1-424a-b896-fdcec82ef324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:42.347228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a4c3c-fba1-424a-b896-fdcec82ef324 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:42.347507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a4c3c-fba1-424a-b896-fdcec82ef324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:42.347778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a4c3c-fba1-424a-b896-fdcec82ef324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:42.348448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a4c3c-fba1-424a-b896-fdcec82ef324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:42.350087 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a4c3c-fba1-424a-b896-fdcec82ef324 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:42.362512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1298/2607] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:42.367075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044d0ed-fbaa-4654-a57c-0317937b71ee admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:42.377383 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044d0ed-fbaa-4654-a57c-0317937b71ee admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:42.382111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb71a3-18a2-4b1e-bee5-b213af2a1e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:42.382759 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb71a3-18a2-4b1e-bee5-b213af2a1e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:42.383045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb71a3-18a2-4b1e-bee5-b213af2a1e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:42.383319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb71a3-18a2-4b1e-bee5-b213af2a1e95 admin admin] PATH_INFO: `/v3/projects/e02df45e344247e1b6ad5e169971a142/users/4f0252d2bd724683bb54c3d148030d9e/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:42.398511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb71a3-18a2-4b1e-bee5-b213af2a1e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02df45e344247e1b6ad5e169971a142, user_id=4f0252d2bd724683bb54c3d148030d9e, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:42.401716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb71a3-18a2-4b1e-bee5-b213af2a1e95 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:42.416411 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1310/2608] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:18:42 2022] PUT /identity/v3/projects/e02df45e344247e1b6ad5e169971a142/users/4f0252d2bd724683bb54c3d148030d9e/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:42.420458 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a4c3c-fba1-424a-b896-fdcec82ef324 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:42.430592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a4c3c-fba1-424a-b896-fdcec82ef324 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:42.486596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb0d66-21b7-4056-831d-63fe026242b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:42.487238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb0d66-21b7-4056-831d-63fe026242b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:42.487470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb0d66-21b7-4056-831d-63fe026242b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:42.487742 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb0d66-21b7-4056-831d-63fe026242b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:42.488379 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb0d66-21b7-4056-831d-63fe026242b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:42.490002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb0d66-21b7-4056-831d-63fe026242b0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:42.499796 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1299/2609] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:42.504039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb71a3-18a2-4b1e-bee5-b213af2a1e95 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:42.515527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb71a3-18a2-4b1e-bee5-b213af2a1e95 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:42.520308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472c64-833f-44ff-8d6a-661a4a35fce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:42.520882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472c64-833f-44ff-8d6a-661a4a35fce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:42.521159 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472c64-833f-44ff-8d6a-661a4a35fce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:42.521433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1472c64-833f-44ff-8d6a-661a4a35fce6 admin admin] PATH_INFO: `/v3/projects/e02df45e344247e1b6ad5e169971a142/users/4f0252d2bd724683bb54c3d148030d9e/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:42.523403 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1472c64-833f-44ff-8d6a-661a4a35fce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e02df45e344247e1b6ad5e169971a142, user_id=4f0252d2bd724683bb54c3d148030d9e, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:42.526166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1472c64-833f-44ff-8d6a-661a4a35fce6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:42.536367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1311/2610] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:18:42 2022] PUT /identity/v3/projects/e02df45e344247e1b6ad5e169971a142/users/4f0252d2bd724683bb54c3d148030d9e/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:18:42.548148 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49111f-71df-49d5-8ac5-3e88658524ac None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:42.548347 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49111f-71df-49d5-8ac5-3e88658524ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:42.548623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49111f-71df-49d5-8ac5-3e88658524ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:42.577052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4a49111f-71df-49d5-8ac5-3e88658524ac None None] MFA Rules not processed for user `4f0252d2bd724683bb54c3d148030d9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:42.578399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4a49111f-71df-49d5-8ac5-3e88658524ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:42.623798 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:42.648553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1300/2611] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:42.660402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472c64-833f-44ff-8d6a-661a4a35fce6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:42.662149 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b1472c64-833f-44ff-8d6a-661a4a35fce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:42.726220 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1472c64-833f-44ff-8d6a-661a4a35fce6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:42.730995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdd190-2c64-4987-9672-bf0d91e1e290 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:42.731603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdd190-2c64-4987-9672-bf0d91e1e290 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:42.731879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdd190-2c64-4987-9672-bf0d91e1e290 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:42.732147 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fdd190-2c64-4987-9672-bf0d91e1e290 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:42.742108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdd190-2c64-4987-9672-bf0d91e1e290 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:42.744342 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fdd190-2c64-4987-9672-bf0d91e1e290 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:42.755156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1312/2612] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:42 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:18:42.905487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49111f-71df-49d5-8ac5-3e88658524ac None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:42.915110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49111f-71df-49d5-8ac5-3e88658524ac None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:42.919635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2787f7-3fe9-4218-8cc9-47b2501f6860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:42.920236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2787f7-3fe9-4218-8cc9-47b2501f6860 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:42.920517 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2787f7-3fe9-4218-8cc9-47b2501f6860 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:42.920793 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2787f7-3fe9-4218-8cc9-47b2501f6860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:42.976825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2787f7-3fe9-4218-8cc9-47b2501f6860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:42.978929 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2787f7-3fe9-4218-8cc9-47b2501f6860 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:42.987917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1301/2613] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:42 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:18:52.381510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdd190-2c64-4987-9672-bf0d91e1e290 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:52.446589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fdd190-2c64-4987-9672-bf0d91e1e290 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:52.451772 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:52.452433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:52.452740 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:52.453017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] PATH_INFO: `/v3/users/e43a2a2e84bb40dc92797b664fbaea44` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:52.454181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43a2a2e84bb40dc92797b664fbaea44)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:52.456185 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:52.504657 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:52.506397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43a2a2e84bb40dc92797b664fbaea44'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:52.507148 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43a2a2e84bb40dc92797b664fbaea44'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:52.508166 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43a2a2e84bb40dc92797b664fbaea44'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:52.518152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43a2a2e84bb40dc92797b664fbaea44'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:52.531052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43a2a2e84bb40dc92797b664fbaea44'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:52.543630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1313/2614] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:18:52 2022] DELETE /identity/v3/users/e43a2a2e84bb40dc92797b664fbaea44 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:52.548462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e4ae-2d14-4677-a3a2-ec1d98ca2549 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:52.548927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e4ae-2d14-4677-a3a2-ec1d98ca2549 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:52.549065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3e4ae-2d14-4677-a3a2-ec1d98ca2549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:52.569544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9fa3e4ae-2d14-4677-a3a2-ec1d98ca2549 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:52.570829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9fa3e4ae-2d14-4677-a3a2-ec1d98ca2549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:52.630772 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1302/2615] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:52.640726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:52.642182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:52.699885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242e102-6010-4d35-8597-c417b13b5282 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:52.703576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d36bc8-0e39-4c55-8bf5-f31ea9064216 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:52.704175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d36bc8-0e39-4c55-8bf5-f31ea9064216 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:52.704482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d36bc8-0e39-4c55-8bf5-f31ea9064216 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:52.704755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d36bc8-0e39-4c55-8bf5-f31ea9064216 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:52.713834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d36bc8-0e39-4c55-8bf5-f31ea9064216 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:52.715329 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d36bc8-0e39-4c55-8bf5-f31ea9064216 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:52.724766 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1314/2616] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:18:52.994417 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3e4ae-2d14-4677-a3a2-ec1d98ca2549 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:52.995851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9fa3e4ae-2d14-4677-a3a2-ec1d98ca2549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:53.010034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3e4ae-2d14-4677-a3a2-ec1d98ca2549 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:53.014730 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:53.016460 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:53.017281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:53.017559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] PATH_INFO: `/v3/projects/bce8455f824f4e69ba1b4e0f38a4196a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:53.018733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce8455f824f4e69ba1b4e0f38a4196a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:53.020621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:53.068704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce8455f824f4e69ba1b4e0f38a4196a', 'request_id': 'req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.074830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce8455f824f4e69ba1b4e0f38a4196a', 'request_id': 'req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.080624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce8455f824f4e69ba1b4e0f38a4196a', 'request_id': 'req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.094585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] The token cache is being invalidate because project bce8455f824f4e69ba1b4e0f38a4196a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:18:53.094888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.095523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.097068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1303/2617] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:18:52 2022] DELETE /identity/v3/projects/bce8455f824f4e69ba1b4e0f38a4196a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:53.101161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d36bc8-0e39-4c55-8bf5-f31ea9064216 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:53.111508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d36bc8-0e39-4c55-8bf5-f31ea9064216 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:53.166963 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:53.167668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:53.167987 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:53.168784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] PATH_INFO: `/v3/users/4f0252d2bd724683bb54c3d148030d9e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:53.169958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0252d2bd724683bb54c3d148030d9e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:53.171882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:53.218571 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0252d2bd724683bb54c3d148030d9e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.219205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0252d2bd724683bb54c3d148030d9e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.219798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0252d2bd724683bb54c3d148030d9e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.227142 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0252d2bd724683bb54c3d148030d9e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.238798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:53.241491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-25ef59c9-d43a-4f4c-baee-afdcd878403d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0252d2bd724683bb54c3d148030d9e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.255910 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1315/2618] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:18:53 2022] DELETE /identity/v3/users/4f0252d2bd724683bb54c3d148030d9e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:53.526077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:53.528250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:53.537326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:53.541736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff94f7-19fc-4d67-96f8-3348d42a9f82 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:53.604484 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:53.605180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:53.605393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:53.605669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] PATH_INFO: `/v3/projects/e02df45e344247e1b6ad5e169971a142` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:53.607038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e02df45e344247e1b6ad5e169971a142)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:53.608952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:53.659122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e02df45e344247e1b6ad5e169971a142', 'request_id': 'req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.665273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e02df45e344247e1b6ad5e169971a142', 'request_id': 'req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.670773 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e02df45e344247e1b6ad5e169971a142', 'request_id': 'req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.684142 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] The token cache is being invalidate because project e02df45e344247e1b6ad5e169971a142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:18:53.684553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.685298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-57d68da0-c702-44f8-ba18-b0ba94f6d3fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:53.686961 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1304/2619] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:18:53 2022] DELETE /identity/v3/projects/e02df45e344247e1b6ad5e169971a142 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:53.692731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b4fb7-f559-4ef9-8f38-247b3e2fea95 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:53.693219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b4fb7-f559-4ef9-8f38-247b3e2fea95 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:53.693316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b4fb7-f559-4ef9-8f38-247b3e2fea95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:53.714065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-091b4fb7-f559-4ef9-8f38-247b3e2fea95 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:53.715076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-091b4fb7-f559-4ef9-8f38-247b3e2fea95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:53.775307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1316/2620] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:18:53.783242 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c3c72-2ba8-4703-bbd6-cea4b62b1edc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:53.783525 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c3c72-2ba8-4703-bbd6-cea4b62b1edc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:53.783803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c3c72-2ba8-4703-bbd6-cea4b62b1edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:53.803382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2b7c3c72-2ba8-4703-bbd6-cea4b62b1edc None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:53.804594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b7c3c72-2ba8-4703-bbd6-cea4b62b1edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:53.813079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1305/2621] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:53.817675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b4fb7-f559-4ef9-8f38-247b3e2fea95 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:53.828018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b4fb7-f559-4ef9-8f38-247b3e2fea95 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:53.832639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2feb2e-f6b8-4f10-a176-552628295b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:53.833245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2feb2e-f6b8-4f10-a176-552628295b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:53.833524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2feb2e-f6b8-4f10-a176-552628295b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:53.833828 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2feb2e-f6b8-4f10-a176-552628295b50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:53.834473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2feb2e-f6b8-4f10-a176-552628295b50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:18:53.834786 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2feb2e-f6b8-4f10-a176-552628295b50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:53.836449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2feb2e-f6b8-4f10-a176-552628295b50 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:53.849279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1317/2622] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:18:53.853640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c3c72-2ba8-4703-bbd6-cea4b62b1edc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:53.863450 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c3c72-2ba8-4703-bbd6-cea4b62b1edc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:53.868393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d3f55-6803-4d21-bf0b-ad2f179b6f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:53.868997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d3f55-6803-4d21-bf0b-ad2f179b6f7d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:53.869334 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d3f55-6803-4d21-bf0b-ad2f179b6f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:53.869611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126d3f55-6803-4d21-bf0b-ad2f179b6f7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:53.870627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d3f55-6803-4d21-bf0b-ad2f179b6f7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:53.873043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126d3f55-6803-4d21-bf0b-ad2f179b6f7d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:53.908346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1306/2623] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:18:53 2022] POST /identity/v3/projects => generated 359 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:18:53.909550 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2feb2e-f6b8-4f10-a176-552628295b50 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:53.925075 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2feb2e-f6b8-4f10-a176-552628295b50 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:53.977651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ae997-74c4-4872-9d2e-35faef660743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:53.978199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ae997-74c4-4872-9d2e-35faef660743 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:53.978477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ae997-74c4-4872-9d2e-35faef660743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:53.978748 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ae997-74c4-4872-9d2e-35faef660743 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:53.979379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ae997-74c4-4872-9d2e-35faef660743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:53.981327 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ae997-74c4-4872-9d2e-35faef660743 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:54.001889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1318/2624] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:18:53 2022] POST /identity/v3/users => generated 280 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:18:54.005983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d3f55-6803-4d21-bf0b-ad2f179b6f7d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:54.018576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126d3f55-6803-4d21-bf0b-ad2f179b6f7d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:54.023249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6df04-ba73-40df-9d53-164ffdea44ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:54.023848 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6df04-ba73-40df-9d53-164ffdea44ab admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:54.024128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6df04-ba73-40df-9d53-164ffdea44ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:54.024404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6df04-ba73-40df-9d53-164ffdea44ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:54.025064 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6df04-ba73-40df-9d53-164ffdea44ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:54.026700 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6df04-ba73-40df-9d53-164ffdea44ab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:54.037432 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1307/2625] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:54.041486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ae997-74c4-4872-9d2e-35faef660743 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:54.052432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ae997-74c4-4872-9d2e-35faef660743 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:54.057183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c71841-acab-4cf5-857c-cbf396b71b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:54.057808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c71841-acab-4cf5-857c-cbf396b71b0e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:54.058086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c71841-acab-4cf5-857c-cbf396b71b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:54.058355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c71841-acab-4cf5-857c-cbf396b71b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:54.059898 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c71841-acab-4cf5-857c-cbf396b71b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:54.061504 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c71841-acab-4cf5-857c-cbf396b71b0e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:54.072060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1319/2626] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:54.076422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6df04-ba73-40df-9d53-164ffdea44ab admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:54.086502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6df04-ba73-40df-9d53-164ffdea44ab admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:54.091552 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80b0e6-1b5f-426a-8e03-30a70a39ee56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:54.092202 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80b0e6-1b5f-426a-8e03-30a70a39ee56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:54.092473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80b0e6-1b5f-426a-8e03-30a70a39ee56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:54.092747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80b0e6-1b5f-426a-8e03-30a70a39ee56 admin admin] PATH_INFO: `/v3/projects/9b2e414a6cfa476995231f63083201b1/users/72d5f66ff4cb4eb788023fc852fda511/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:54.108034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80b0e6-1b5f-426a-8e03-30a70a39ee56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b2e414a6cfa476995231f63083201b1, user_id=72d5f66ff4cb4eb788023fc852fda511, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:54.110832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80b0e6-1b5f-426a-8e03-30a70a39ee56 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:54.124670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1308/2627] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:18:54 2022] PUT /identity/v3/projects/9b2e414a6cfa476995231f63083201b1/users/72d5f66ff4cb4eb788023fc852fda511/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:54.136059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4c634-011f-4a83-be0f-418b73b4415e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:54.136346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4c634-011f-4a83-be0f-418b73b4415e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:54.136633 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4c634-011f-4a83-be0f-418b73b4415e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:54.164281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-54a4c634-011f-4a83-be0f-418b73b4415e None None] MFA Rules not processed for user `72d5f66ff4cb4eb788023fc852fda511`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:54.165509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-54a4c634-011f-4a83-be0f-418b73b4415e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:54.208712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:54.235038 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1320/2628] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:54.269575 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80b0e6-1b5f-426a-8e03-30a70a39ee56 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:54.271344 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-af80b0e6-1b5f-426a-8e03-30a70a39ee56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:54.305126 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:54.331067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80b0e6-1b5f-426a-8e03-30a70a39ee56 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:54.334676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4545e65-fcbf-4f8e-9209-0311527c5301 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:54.335280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4545e65-fcbf-4f8e-9209-0311527c5301 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:54.335585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4545e65-fcbf-4f8e-9209-0311527c5301 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:54.335853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4545e65-fcbf-4f8e-9209-0311527c5301 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:54.344595 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4545e65-fcbf-4f8e-9209-0311527c5301 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:54.346049 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4545e65-fcbf-4f8e-9209-0311527c5301 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:54.354846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1309/2629] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:54 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:18:54.370292 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4c634-011f-4a83-be0f-418b73b4415e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:54.432846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4c634-011f-4a83-be0f-418b73b4415e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:54.438091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:54.438717 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:54.438993 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:54.439271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] PATH_INFO: `/v3/users/72d5f66ff4cb4eb788023fc852fda511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:54.440402 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d5f66ff4cb4eb788023fc852fda511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:54.442385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:54.489409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d5f66ff4cb4eb788023fc852fda511'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:54.490005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d5f66ff4cb4eb788023fc852fda511'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:54.490616 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d5f66ff4cb4eb788023fc852fda511'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:54.499128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d5f66ff4cb4eb788023fc852fda511'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:54.511278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d5f66ff4cb4eb788023fc852fda511'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:54.523752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1321/2630] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:18:54 2022] DELETE /identity/v3/users/72d5f66ff4cb4eb788023fc852fda511 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:54.529337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2fcd8-8f0c-4a1d-b244-9ea210e3f41d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:54.529635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2fcd8-8f0c-4a1d-b244-9ea210e3f41d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:54.530034 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2fcd8-8f0c-4a1d-b244-9ea210e3f41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:54.551228 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-55d2fcd8-8f0c-4a1d-b244-9ea210e3f41d None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:54.552438 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-55d2fcd8-8f0c-4a1d-b244-9ea210e3f41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:54.614270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1310/2631] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:54.624212 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:54.625656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:54.685508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d27110a-ef5b-4404-ae13-42e859ae8495 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:54.689211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af12f6-176c-49b1-b400-20536cee2291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:54.689850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af12f6-176c-49b1-b400-20536cee2291 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:54.690130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af12f6-176c-49b1-b400-20536cee2291 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:54.690404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af12f6-176c-49b1-b400-20536cee2291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:54.700078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af12f6-176c-49b1-b400-20536cee2291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:54.701512 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af12f6-176c-49b1-b400-20536cee2291 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:54.711108 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1322/2632] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:18:54.990559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2fcd8-8f0c-4a1d-b244-9ea210e3f41d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:54.992174 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-55d2fcd8-8f0c-4a1d-b244-9ea210e3f41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:55.004917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2fcd8-8f0c-4a1d-b244-9ea210e3f41d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:55.009552 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:55.010214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.010494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.010767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] PATH_INFO: `/v3/projects/9b2e414a6cfa476995231f63083201b1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.012786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b2e414a6cfa476995231f63083201b1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:55.014763 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:55.063304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b2e414a6cfa476995231f63083201b1', 'request_id': 'req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:55.070127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b2e414a6cfa476995231f63083201b1', 'request_id': 'req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:55.076299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b2e414a6cfa476995231f63083201b1', 'request_id': 'req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:55.089406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] The token cache is being invalidate because project 9b2e414a6cfa476995231f63083201b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:18:55.089688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:55.091044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-26553fe6-44cc-4e4f-9cbf-b869a9b1872f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:18:55.092539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1311/2633] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:18:54 2022] DELETE /identity/v3/projects/9b2e414a6cfa476995231f63083201b1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:55.098441 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e12af-f1e2-4740-a366-ebad6031ca09 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.098726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e12af-f1e2-4740-a366-ebad6031ca09 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.099002 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135e12af-f1e2-4740-a366-ebad6031ca09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.106590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:55.119659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-135e12af-f1e2-4740-a366-ebad6031ca09 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:55.121420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-135e12af-f1e2-4740-a366-ebad6031ca09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:55.181986 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1323/2634] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:55.190628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921f490-a6f4-40a3-94f0-dcd554411f73 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.190916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921f490-a6f4-40a3-94f0-dcd554411f73 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.191219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921f490-a6f4-40a3-94f0-dcd554411f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.211619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f921f490-a6f4-40a3-94f0-dcd554411f73 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:55.212843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f921f490-a6f4-40a3-94f0-dcd554411f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:55.221060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1312/2635] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:55.225295 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e12af-f1e2-4740-a366-ebad6031ca09 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:55.235169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135e12af-f1e2-4740-a366-ebad6031ca09 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:55.239911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1b019-579f-408c-9d53-5db0bf75d457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:55.240514 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1b019-579f-408c-9d53-5db0bf75d457 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.240793 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1b019-579f-408c-9d53-5db0bf75d457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.241058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1b019-579f-408c-9d53-5db0bf75d457 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.241722 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1b019-579f-408c-9d53-5db0bf75d457 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:18:55.242070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1b019-579f-408c-9d53-5db0bf75d457 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:55.243638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1b019-579f-408c-9d53-5db0bf75d457 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:55.256111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1324/2636] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:18:55.260490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921f490-a6f4-40a3-94f0-dcd554411f73 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:55.270667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921f490-a6f4-40a3-94f0-dcd554411f73 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:55.275315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674580e5-3965-4795-bdfc-aba3f2e0c5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:55.275935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674580e5-3965-4795-bdfc-aba3f2e0c5db admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.276215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674580e5-3965-4795-bdfc-aba3f2e0c5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.276489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674580e5-3965-4795-bdfc-aba3f2e0c5db admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.277122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674580e5-3965-4795-bdfc-aba3f2e0c5db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:55.279286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674580e5-3965-4795-bdfc-aba3f2e0c5db admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:55.305756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1313/2637] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:18:55 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:18:55.311102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1b019-579f-408c-9d53-5db0bf75d457 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:55.321240 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1b019-579f-408c-9d53-5db0bf75d457 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:55.377252 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a678b1a-af1e-46f6-a1a4-1b84ebe05056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:55.377919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a678b1a-af1e-46f6-a1a4-1b84ebe05056 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.378199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a678b1a-af1e-46f6-a1a4-1b84ebe05056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.378473 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a678b1a-af1e-46f6-a1a4-1b84ebe05056 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.379095 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a678b1a-af1e-46f6-a1a4-1b84ebe05056 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:55.381018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a678b1a-af1e-46f6-a1a4-1b84ebe05056 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:55.403864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1325/2638] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:18:55 2022] POST /identity/v3/users => generated 295 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:18:55.407143 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674580e5-3965-4795-bdfc-aba3f2e0c5db admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:55.417266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674580e5-3965-4795-bdfc-aba3f2e0c5db admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:55.422810 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4f0d0-80e4-4e0f-ae4d-fe961d98d24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:55.423759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4f0d0-80e4-4e0f-ae4d-fe961d98d24e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.424219 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4f0d0-80e4-4e0f-ae4d-fe961d98d24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.424493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4f0d0-80e4-4e0f-ae4d-fe961d98d24e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.425130 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4f0d0-80e4-4e0f-ae4d-fe961d98d24e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:55.426770 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4f0d0-80e4-4e0f-ae4d-fe961d98d24e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:55.437545 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1314/2639] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:55.441645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a678b1a-af1e-46f6-a1a4-1b84ebe05056 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:55.452959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a678b1a-af1e-46f6-a1a4-1b84ebe05056 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:55.457882 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3d117-dd33-4914-ab21-5210606691de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:55.458497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3d117-dd33-4914-ab21-5210606691de admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.458787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3d117-dd33-4914-ab21-5210606691de admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.459063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3d117-dd33-4914-ab21-5210606691de admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.459712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3d117-dd33-4914-ab21-5210606691de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:55.461315 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b3d117-dd33-4914-ab21-5210606691de admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:55.472261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1326/2640] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:18:55.476496 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4f0d0-80e4-4e0f-ae4d-fe961d98d24e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:55.486639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4f0d0-80e4-4e0f-ae4d-fe961d98d24e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:55.491643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9373ce-8838-4c3e-9617-b0406e3d53a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:55.492289 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9373ce-8838-4c3e-9617-b0406e3d53a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.492571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9373ce-8838-4c3e-9617-b0406e3d53a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.492849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9373ce-8838-4c3e-9617-b0406e3d53a2 admin admin] PATH_INFO: `/v3/projects/304618300e864cfbbcf90ebbb501bef7/users/e1429676b9ff44dfb38632636ed92600/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.507588 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9373ce-8838-4c3e-9617-b0406e3d53a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=304618300e864cfbbcf90ebbb501bef7, user_id=e1429676b9ff44dfb38632636ed92600, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:55.510284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9373ce-8838-4c3e-9617-b0406e3d53a2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:55.513100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:55.526205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1315/2641] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:18:55 2022] PUT /identity/v3/projects/304618300e864cfbbcf90ebbb501bef7/users/e1429676b9ff44dfb38632636ed92600/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:18:55.537841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b075da-0e12-451c-a2cd-8c01ff57cbeb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.538211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b075da-0e12-451c-a2cd-8c01ff57cbeb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.538551 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b075da-0e12-451c-a2cd-8c01ff57cbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.568592 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-35b075da-0e12-451c-a2cd-8c01ff57cbeb None None] MFA Rules not processed for user `e1429676b9ff44dfb38632636ed92600`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:18:55.569857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-35b075da-0e12-451c-a2cd-8c01ff57cbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:55.630120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:18:55.635961 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1327/2642] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:18:55 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:18:55.646924 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9373ce-8838-4c3e-9617-b0406e3d53a2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:18:55.648696 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fd9373ce-8838-4c3e-9617-b0406e3d53a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:18:55.708112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9373ce-8838-4c3e-9617-b0406e3d53a2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:18:55.711724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b96b9-14b1-497a-9979-aec02a9968a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:18:55.712324 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b96b9-14b1-497a-9979-aec02a9968a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:18:55.712609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b96b9-14b1-497a-9979-aec02a9968a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:18:55.713038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b96b9-14b1-497a-9979-aec02a9968a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:18:55.721574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b96b9-14b1-497a-9979-aec02a9968a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:18:55.723036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b96b9-14b1-497a-9979-aec02a9968a3 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:18:55.733483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1316/2643] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:18:55 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:05.636934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:05.727935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=1,keystone:SELECT=28 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:14.049371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b075da-0e12-451c-a2cd-8c01ff57cbeb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:14.050914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-35b075da-0e12-451c-a2cd-8c01ff57cbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:14.152847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b075da-0e12-451c-a2cd-8c01ff57cbeb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:14.159018 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f005b-bf23-45e7-bbe2-f344706dff35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:14.159723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f005b-bf23-45e7-bbe2-f344706dff35 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:14.160010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f005b-bf23-45e7-bbe2-f344706dff35 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:14.160288 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f005b-bf23-45e7-bbe2-f344706dff35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:14.163205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8c9f005b-bf23-45e7-bbe2-f344706dff35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:14.199071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f005b-bf23-45e7-bbe2-f344706dff35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:14.201630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f005b-bf23-45e7-bbe2-f344706dff35 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:14.214650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1328/2644] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:18.271159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b96b9-14b1-497a-9979-aec02a9968a3 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:18.334390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b96b9-14b1-497a-9979-aec02a9968a3 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:18.339071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:18.339885 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:18.340033 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:18.340318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] PATH_INFO: `/v3/users/e1429676b9ff44dfb38632636ed92600` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:18.341660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1429676b9ff44dfb38632636ed92600)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:18.343680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:18.394775 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1429676b9ff44dfb38632636ed92600'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:18.406994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1429676b9ff44dfb38632636ed92600'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:18.407621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1429676b9ff44dfb38632636ed92600'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:18.408235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1429676b9ff44dfb38632636ed92600'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:18.416256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1429676b9ff44dfb38632636ed92600'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:18.435295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1317/2645] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:19:18 2022] DELETE /identity/v3/users/e1429676b9ff44dfb38632636ed92600 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:18.441838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60effd8b-c96d-4670-ba64-9628f12cf3fe None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:18.441838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60effd8b-c96d-4670-ba64-9628f12cf3fe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:18.441838 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60effd8b-c96d-4670-ba64-9628f12cf3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:18.462231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-60effd8b-c96d-4670-ba64-9628f12cf3fe None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:18.463428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-60effd8b-c96d-4670-ba64-9628f12cf3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:18.543991 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1329/2646] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:18.553656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:18.555118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:18.626681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb852ae-2964-4de4-b25f-36ca8975618c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:18.630407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2cc44-5056-4d82-a51d-ab90aa8f8b8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:18.631026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2cc44-5056-4d82-a51d-ab90aa8f8b8d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:18.631299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2cc44-5056-4d82-a51d-ab90aa8f8b8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:18.631571 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2cc44-5056-4d82-a51d-ab90aa8f8b8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:18.642284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2cc44-5056-4d82-a51d-ab90aa8f8b8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:18.643751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2cc44-5056-4d82-a51d-ab90aa8f8b8d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:18.654529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1318/2647] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:18.952211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60effd8b-c96d-4670-ba64-9628f12cf3fe None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:18.953756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-60effd8b-c96d-4670-ba64-9628f12cf3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:18.967724 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60effd8b-c96d-4670-ba64-9628f12cf3fe None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:18.972565 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:18.973312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:18.973591 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:18.973913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] PATH_INFO: `/v3/projects/304618300e864cfbbcf90ebbb501bef7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:18.975048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=304618300e864cfbbcf90ebbb501bef7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:18.976967 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:19.029076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304618300e864cfbbcf90ebbb501bef7', 'request_id': 'req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:19.036548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304618300e864cfbbcf90ebbb501bef7', 'request_id': 'req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:19.042221 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304618300e864cfbbcf90ebbb501bef7', 'request_id': 'req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:19.053457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] The token cache is being invalidate because project 304618300e864cfbbcf90ebbb501bef7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:19:19.053742 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:19.054408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ea3ea60-775d-4ecb-a547-fe0e9e8dceb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:19.055889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1330/2648] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:19:18 2022] DELETE /identity/v3/projects/304618300e864cfbbcf90ebbb501bef7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:19.061237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775ace5-58f6-4acb-b4eb-bea5a71d176b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.061520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775ace5-58f6-4acb-b4eb-bea5a71d176b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.061814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a775ace5-58f6-4acb-b4eb-bea5a71d176b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.081343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a775ace5-58f6-4acb-b4eb-bea5a71d176b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:19.082601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a775ace5-58f6-4acb-b4eb-bea5a71d176b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:19.100367 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:19.144589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1319/2649] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:19.153368 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35e3f0-7cce-4072-98cf-2156fe127ed1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.153580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35e3f0-7cce-4072-98cf-2156fe127ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.153883 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35e3f0-7cce-4072-98cf-2156fe127ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.174042 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6c35e3f0-7cce-4072-98cf-2156fe127ed1 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:19.175234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6c35e3f0-7cce-4072-98cf-2156fe127ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:19.183955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1331/2650] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:19.188532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775ace5-58f6-4acb-b4eb-bea5a71d176b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:19.198748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a775ace5-58f6-4acb-b4eb-bea5a71d176b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:19.204281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b5562-d703-4cf9-9a1a-1a55c1ef9f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:19.204932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b5562-d703-4cf9-9a1a-1a55c1ef9f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.205213 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b5562-d703-4cf9-9a1a-1a55c1ef9f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.205500 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b5562-d703-4cf9-9a1a-1a55c1ef9f25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.206171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b5562-d703-4cf9-9a1a-1a55c1ef9f25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:19:19.206472 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b5562-d703-4cf9-9a1a-1a55c1ef9f25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:19.208068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b5562-d703-4cf9-9a1a-1a55c1ef9f25 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:19.220081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1320/2651] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:19:19.224444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35e3f0-7cce-4072-98cf-2156fe127ed1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:19.228877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:19.235372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35e3f0-7cce-4072-98cf-2156fe127ed1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:19.241506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53710e44-8b74-44fc-a3e9-e2e0dd983c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:19.242155 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53710e44-8b74-44fc-a3e9-e2e0dd983c66 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.242432 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53710e44-8b74-44fc-a3e9-e2e0dd983c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.242705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53710e44-8b74-44fc-a3e9-e2e0dd983c66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.243335 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53710e44-8b74-44fc-a3e9-e2e0dd983c66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:19.245227 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53710e44-8b74-44fc-a3e9-e2e0dd983c66 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:19.274548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1332/2652] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:19 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:19.279003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b5562-d703-4cf9-9a1a-1a55c1ef9f25 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:19.289251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b5562-d703-4cf9-9a1a-1a55c1ef9f25 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:19.349898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e69d7-275f-4aee-b89b-b747c552691e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:19.350504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e69d7-275f-4aee-b89b-b747c552691e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.350783 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e69d7-275f-4aee-b89b-b747c552691e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.351081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2e69d7-275f-4aee-b89b-b747c552691e admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.351704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e69d7-275f-4aee-b89b-b747c552691e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:19.353623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2e69d7-275f-4aee-b89b-b747c552691e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:19.375464 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1321/2653] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:19 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:19.379566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53710e44-8b74-44fc-a3e9-e2e0dd983c66 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:19.390817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53710e44-8b74-44fc-a3e9-e2e0dd983c66 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:19.395955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f2bd1-c9a9-461f-9e32-d4f68c84567c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:19.396464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f2bd1-c9a9-461f-9e32-d4f68c84567c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.396744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f2bd1-c9a9-461f-9e32-d4f68c84567c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.397015 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f2bd1-c9a9-461f-9e32-d4f68c84567c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.397656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f2bd1-c9a9-461f-9e32-d4f68c84567c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:19.399308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f2bd1-c9a9-461f-9e32-d4f68c84567c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:19.410124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1333/2654] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:19.414362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e69d7-275f-4aee-b89b-b747c552691e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:19.424494 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2e69d7-275f-4aee-b89b-b747c552691e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:19.425350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f2bd1-c9a9-461f-9e32-d4f68c84567c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:19.427258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ec6f2bd1-c9a9-461f-9e32-d4f68c84567c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:19.429191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3348a4c-47b1-454c-971e-f72c6a63d1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:19.429854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3348a4c-47b1-454c-971e-f72c6a63d1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.430138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3348a4c-47b1-454c-971e-f72c6a63d1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.430404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3348a4c-47b1-454c-971e-f72c6a63d1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.431040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3348a4c-47b1-454c-971e-f72c6a63d1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:19.432627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3348a4c-47b1-454c-971e-f72c6a63d1d1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:19.445357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1322/2655] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:19.450180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3348a4c-47b1-454c-971e-f72c6a63d1d1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:19.460404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3348a4c-47b1-454c-971e-f72c6a63d1d1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:19.466205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6aedd-c509-4998-bca6-d6db81f381a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:19.466855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6aedd-c509-4998-bca6-d6db81f381a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.467136 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6aedd-c509-4998-bca6-d6db81f381a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.467410 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6aedd-c509-4998-bca6-d6db81f381a0 admin admin] PATH_INFO: `/v3/projects/b5c70daaa3b340a286f8f4ebe26b807d/users/d2749b3c8da94ce1b4af6c708a2f9db6/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.482977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6aedd-c509-4998-bca6-d6db81f381a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c70daaa3b340a286f8f4ebe26b807d, user_id=d2749b3c8da94ce1b4af6c708a2f9db6, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:19.486701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6aedd-c509-4998-bca6-d6db81f381a0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:19.489454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f2bd1-c9a9-461f-9e32-d4f68c84567c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:19.493644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a3881-d63c-4be4-bb8f-68e59c6218b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:19.494388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a3881-d63c-4be4-bb8f-68e59c6218b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.494745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a3881-d63c-4be4-bb8f-68e59c6218b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.495116 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6a3881-d63c-4be4-bb8f-68e59c6218b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.496970 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6d6a3881-d63c-4be4-bb8f-68e59c6218b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:19.500196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1323/2656] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:19 2022] PUT /identity/v3/projects/b5c70daaa3b340a286f8f4ebe26b807d/users/d2749b3c8da94ce1b4af6c708a2f9db6/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:19:19.505418 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd6d46-c434-477a-bc02-d38058155f21 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.505704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd6d46-c434-477a-bc02-d38058155f21 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.506088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dd6d46-c434-477a-bc02-d38058155f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.510922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a3881-d63c-4be4-bb8f-68e59c6218b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:19.512412 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6a3881-d63c-4be4-bb8f-68e59c6218b2 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:19.521461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1334/2657] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:19:19.525196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-82dd6d46-c434-477a-bc02-d38058155f21 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:19.526463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-82dd6d46-c434-477a-bc02-d38058155f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:19.592994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1324/2658] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:19.603140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a3881-d63c-4be4-bb8f-68e59c6218b2 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:19.604586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6d6a3881-d63c-4be4-bb8f-68e59c6218b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:19.615785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6a3881-d63c-4be4-bb8f-68e59c6218b2 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:19.672313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e9cdc-34bb-48f1-8ea6-e082b5ee57ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:19.673017 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e9cdc-34bb-48f1-8ea6-e082b5ee57ea service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.673806 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e9cdc-34bb-48f1-8ea6-e082b5ee57ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.673965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e9cdc-34bb-48f1-8ea6-e082b5ee57ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.686901 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e9cdc-34bb-48f1-8ea6-e082b5ee57ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:19.688357 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e9cdc-34bb-48f1-8ea6-e082b5ee57ea service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:19.699599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1335/2659] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:19.759486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd6d46-c434-477a-bc02-d38058155f21 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:19.766981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:19.771686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dd6d46-c434-477a-bc02-d38058155f21 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:19.776801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8265f-27f1-4725-a9d8-6dca600fd003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:19.778009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8265f-27f1-4725-a9d8-6dca600fd003 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:19.778294 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8265f-27f1-4725-a9d8-6dca600fd003 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:19.778577 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8265f-27f1-4725-a9d8-6dca600fd003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:19.780383 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b5e8265f-27f1-4725-a9d8-6dca600fd003 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:19.797949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8265f-27f1-4725-a9d8-6dca600fd003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:19.800152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e8265f-27f1-4725-a9d8-6dca600fd003 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:19.816760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1325/2660] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:24.791397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e9cdc-34bb-48f1-8ea6-e082b5ee57ea service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:24.794245 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-929e9cdc-34bb-48f1-8ea6-e082b5ee57ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:24.813117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e9cdc-34bb-48f1-8ea6-e082b5ee57ea service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:24.818923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:24.819851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:24.820354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:24.820955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] PATH_INFO: `/v3/users/7f46019e78cc4e89bf877a4f5a7a8fe9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:24.841728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f46019e78cc4e89bf877a4f5a7a8fe9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:24.844117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:24.916637 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f46019e78cc4e89bf877a4f5a7a8fe9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:24.917539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f46019e78cc4e89bf877a4f5a7a8fe9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:24.919111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f46019e78cc4e89bf877a4f5a7a8fe9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:24.930076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f46019e78cc4e89bf877a4f5a7a8fe9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:24.936852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:24.955178 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f46019e78cc4e89bf877a4f5a7a8fe9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:24.983590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1336/2661] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:19:24 2022] DELETE /identity/v3/users/7f46019e78cc4e89bf877a4f5a7a8fe9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:25.060760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d51e1-2adb-4b84-b2b3-9d3f7ec24700 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.061320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d51e1-2adb-4b84-b2b3-9d3f7ec24700 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.067745 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825d51e1-2adb-4b84-b2b3-9d3f7ec24700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.110832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-825d51e1-2adb-4b84-b2b3-9d3f7ec24700 None None] MFA Rules not processed for user `d2749b3c8da94ce1b4af6c708a2f9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:25.112071 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-825d51e1-2adb-4b84-b2b3-9d3f7ec24700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:25.195025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1326/2662] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:25.200547 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.202057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:25.206557 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d51e1-2adb-4b84-b2b3-9d3f7ec24700 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.208331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-825d51e1-2adb-4b84-b2b3-9d3f7ec24700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:25.327236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825d51e1-2adb-4b84-b2b3-9d3f7ec24700 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.331231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f83edf-c556-49d2-a92c-6b443ba0acd4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.334375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.335335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.335781 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.336216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] PATH_INFO: `/v3/projects/358e50f9b9df43b7aa0a4ba1353e4a25` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.336586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8b729-d952-4a77-9ddf-e6c03e1716d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.337294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8b729-d952-4a77-9ddf-e6c03e1716d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.337876 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8b729-d952-4a77-9ddf-e6c03e1716d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.338260 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8b729-d952-4a77-9ddf-e6c03e1716d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.339008 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8b729-d952-4a77-9ddf-e6c03e1716d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.342903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8b729-d952-4a77-9ddf-e6c03e1716d3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:25.363981 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=358e50f9b9df43b7aa0a4ba1353e4a25)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.367195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:25.398952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1337/2663] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:25 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:25.400316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8b729-d952-4a77-9ddf-e6c03e1716d3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.435558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '358e50f9b9df43b7aa0a4ba1353e4a25', 'request_id': 'req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:25.451935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '358e50f9b9df43b7aa0a4ba1353e4a25', 'request_id': 'req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:25.462454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '358e50f9b9df43b7aa0a4ba1353e4a25', 'request_id': 'req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:25.487393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] The token cache is being invalidate because project 358e50f9b9df43b7aa0a4ba1353e4a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:19:25.488027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:25.489184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef38a45d-4d4a-4034-a3fe-86de81f12f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:25.491983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1327/2664] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:19:25 2022] DELETE /identity/v3/projects/358e50f9b9df43b7aa0a4ba1353e4a25 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:25.499374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490aac9-c28f-4313-8a3e-158af0fcd510 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.499714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490aac9-c28f-4313-8a3e-158af0fcd510 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.500003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490aac9-c28f-4313-8a3e-158af0fcd510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.503522 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8b729-d952-4a77-9ddf-e6c03e1716d3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.508646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:25.524045 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4490aac9-c28f-4313-8a3e-158af0fcd510 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:25.525318 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4490aac9-c28f-4313-8a3e-158af0fcd510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:25.582866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2511af-338f-4a1b-8536-016ced4583a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.583586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2511af-338f-4a1b-8536-016ced4583a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.584219 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2511af-338f-4a1b-8536-016ced4583a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.584502 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2511af-338f-4a1b-8536-016ced4583a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.585457 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2511af-338f-4a1b-8536-016ced4583a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.587842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2511af-338f-4a1b-8536-016ced4583a0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:25.603851 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1328/2665] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:25.607023 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:25.612676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a69dad-dd96-4a57-9a3f-fa51a550f341 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.613211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a69dad-dd96-4a57-9a3f-fa51a550f341 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.613597 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a69dad-dd96-4a57-9a3f-fa51a550f341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.617678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1338/2666] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:25 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:25.619207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2511af-338f-4a1b-8536-016ced4583a0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.622631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ed2511af-338f-4a1b-8536-016ced4583a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:25.638093 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-32a69dad-dd96-4a57-9a3f-fa51a550f341 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:25.638348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2511af-338f-4a1b-8536-016ced4583a0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.639502 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-32a69dad-dd96-4a57-9a3f-fa51a550f341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:25.644320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ca273-eca8-487d-8e6a-319f7fc1cf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.644956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ca273-eca8-487d-8e6a-319f7fc1cf76 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.645239 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ca273-eca8-487d-8e6a-319f7fc1cf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.645515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ca273-eca8-487d-8e6a-319f7fc1cf76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.646215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ca273-eca8-487d-8e6a-319f7fc1cf76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.648047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ca273-eca8-487d-8e6a-319f7fc1cf76 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:25.653247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1329/2667] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:25.656580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a69dad-dd96-4a57-9a3f-fa51a550f341 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.663889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1339/2668] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:25.667589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ca273-eca8-487d-8e6a-319f7fc1cf76 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.673402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a69dad-dd96-4a57-9a3f-fa51a550f341 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.678452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ca273-eca8-487d-8e6a-319f7fc1cf76 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.679389 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39c700-c469-4668-93a8-a12faa58130d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.680262 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39c700-c469-4668-93a8-a12faa58130d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.680642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39c700-c469-4668-93a8-a12faa58130d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.681217 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39c700-c469-4668-93a8-a12faa58130d admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.682083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39c700-c469-4668-93a8-a12faa58130d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:19:25.682579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39c700-c469-4668-93a8-a12faa58130d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.685144 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39c700-c469-4668-93a8-a12faa58130d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:25.690769 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84016a3-05ba-42ea-a96e-6116727dc80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.691452 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84016a3-05ba-42ea-a96e-6116727dc80a admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.691639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84016a3-05ba-42ea-a96e-6116727dc80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.691919 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84016a3-05ba-42ea-a96e-6116727dc80a admin admin] PATH_INFO: `/v3/projects/1c51d69ad366402f977fea46158bcfa4/users/4320be012bb045afb47948c6c7217a70/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.707316 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1330/2669] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:19:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:19:25.708542 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39c700-c469-4668-93a8-a12faa58130d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.716458 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84016a3-05ba-42ea-a96e-6116727dc80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c51d69ad366402f977fea46158bcfa4, user_id=4320be012bb045afb47948c6c7217a70, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.720291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84016a3-05ba-42ea-a96e-6116727dc80a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:25.726665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39c700-c469-4668-93a8-a12faa58130d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.740157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6596c1-8682-4aa0-a5e0-8a7c1e9c3355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.740912 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6596c1-8682-4aa0-a5e0-8a7c1e9c3355 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.741263 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6596c1-8682-4aa0-a5e0-8a7c1e9c3355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.741605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6596c1-8682-4aa0-a5e0-8a7c1e9c3355 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.742414 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6596c1-8682-4aa0-a5e0-8a7c1e9c3355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.745629 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1340/2670] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:25 2022] PUT /identity/v3/projects/1c51d69ad366402f977fea46158bcfa4/users/4320be012bb045afb47948c6c7217a70/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:19:25.745820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6596c1-8682-4aa0-a5e0-8a7c1e9c3355 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:25.747781 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84016a3-05ba-42ea-a96e-6116727dc80a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.799052 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1331/2671] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:25 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:25.804253 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6596c1-8682-4aa0-a5e0-8a7c1e9c3355 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.846525 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84016a3-05ba-42ea-a96e-6116727dc80a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.920972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6596c1-8682-4aa0-a5e0-8a7c1e9c3355 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.926822 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71a7b5-e1ed-4778-a723-83c6a96d99f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.927780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a7b5-e1ed-4778-a723-83c6a96d99f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.928157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a7b5-e1ed-4778-a723-83c6a96d99f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.928608 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de71a7b5-e1ed-4778-a723-83c6a96d99f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.929477 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71a7b5-e1ed-4778-a723-83c6a96d99f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.931611 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de71a7b5-e1ed-4778-a723-83c6a96d99f7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:25.936311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c626e-d70d-4f98-bf75-71c97230dd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.936932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c626e-d70d-4f98-bf75-71c97230dd81 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.937209 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c626e-d70d-4f98-bf75-71c97230dd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.937480 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c626e-d70d-4f98-bf75-71c97230dd81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.938167 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c626e-d70d-4f98-bf75-71c97230dd81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.939777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c626e-d70d-4f98-bf75-71c97230dd81 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:25.958385 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1341/2672] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:25.964058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c626e-d70d-4f98-bf75-71c97230dd81 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.970374 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1332/2673] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:25 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:19:25.973860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71a7b5-e1ed-4778-a723-83c6a96d99f7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:25.976446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c626e-d70d-4f98-bf75-71c97230dd81 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.981506 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2d9fa-9b0f-4072-a544-3acbb2527d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.982237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2d9fa-9b0f-4072-a544-3acbb2527d6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.982515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2d9fa-9b0f-4072-a544-3acbb2527d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.982953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2d9fa-9b0f-4072-a544-3acbb2527d6c admin admin] PATH_INFO: `/v3/projects/1c51d69ad366402f977fea46158bcfa4/users/4320be012bb045afb47948c6c7217a70/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.985005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2d9fa-9b0f-4072-a544-3acbb2527d6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c51d69ad366402f977fea46158bcfa4, user_id=4320be012bb045afb47948c6c7217a70, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.986310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de71a7b5-e1ed-4778-a723-83c6a96d99f7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:25.987736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2d9fa-9b0f-4072-a544-3acbb2527d6c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:25.991721 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddfbdb-7604-4b83-b50f-4cd00b3094f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:25.992351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddfbdb-7604-4b83-b50f-4cd00b3094f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:25.992632 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddfbdb-7604-4b83-b50f-4cd00b3094f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:25.993140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ddfbdb-7604-4b83-b50f-4cd00b3094f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:25.994055 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddfbdb-7604-4b83-b50f-4cd00b3094f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:25.995806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ddfbdb-7604-4b83-b50f-4cd00b3094f4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:26.005503 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1342/2674] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:25 2022] PUT /identity/v3/projects/1c51d69ad366402f977fea46158bcfa4/users/4320be012bb045afb47948c6c7217a70/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:26.007331 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1333/2675] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:26.016778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2d9fa-9b0f-4072-a544-3acbb2527d6c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:26.092061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:26.107241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2d9fa-9b0f-4072-a544-3acbb2527d6c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:26.113406 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd7ed0e-7d00-4e31-a5cf-0204a904373a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:26.114169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7ed0e-7d00-4e31-a5cf-0204a904373a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:26.114455 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7ed0e-7d00-4e31-a5cf-0204a904373a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:26.114727 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd7ed0e-7d00-4e31-a5cf-0204a904373a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:26.115426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7ed0e-7d00-4e31-a5cf-0204a904373a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:26.117210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd7ed0e-7d00-4e31-a5cf-0204a904373a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:26.129255 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1343/2676] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:26.133891 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddfbdb-7604-4b83-b50f-4cd00b3094f4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:26.145902 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ddfbdb-7604-4b83-b50f-4cd00b3094f4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:26.152771 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b7a41-6830-41b4-8db7-f822d8908fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:26.153549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b7a41-6830-41b4-8db7-f822d8908fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:26.154104 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b7a41-6830-41b4-8db7-f822d8908fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:26.154457 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b7a41-6830-41b4-8db7-f822d8908fe5 admin admin] PATH_INFO: `/v3/projects/c2e9ecda937c4259bf440a2fe303d5c5/users/362c9656f9b14fed96b930aa599cd480/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:26.171700 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b7a41-6830-41b4-8db7-f822d8908fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2e9ecda937c4259bf440a2fe303d5c5, user_id=362c9656f9b14fed96b930aa599cd480, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:26.174921 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b7a41-6830-41b4-8db7-f822d8908fe5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:26.193558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1334/2677] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:26 2022] PUT /identity/v3/projects/c2e9ecda937c4259bf440a2fe303d5c5/users/362c9656f9b14fed96b930aa599cd480/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:26.204806 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10365c-86a0-49ed-b228-2d2b20f63a3d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:26.205088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10365c-86a0-49ed-b228-2d2b20f63a3d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:26.205354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10365c-86a0-49ed-b228-2d2b20f63a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:26.251847 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-df10365c-86a0-49ed-b228-2d2b20f63a3d None None] MFA Rules not processed for user `362c9656f9b14fed96b930aa599cd480`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:26.253224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-df10365c-86a0-49ed-b228-2d2b20f63a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:26.332223 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1344/2678] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:26.343786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b7a41-6830-41b4-8db7-f822d8908fe5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:26.345567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-339b7a41-6830-41b4-8db7-f822d8908fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:26.368399 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:26.459186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b7a41-6830-41b4-8db7-f822d8908fe5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:26.467152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f86f3-6360-4557-baba-5fd6534e7e08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:26.468129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f86f3-6360-4557-baba-5fd6534e7e08 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:26.469311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f86f3-6360-4557-baba-5fd6534e7e08 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:26.469823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f86f3-6360-4557-baba-5fd6534e7e08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:26.481612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f86f3-6360-4557-baba-5fd6534e7e08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:26.484250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f86f3-6360-4557-baba-5fd6534e7e08 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:26.498804 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1335/2679] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:26.755092 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10365c-86a0-49ed-b228-2d2b20f63a3d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:26.838006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10365c-86a0-49ed-b228-2d2b20f63a3d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:26.847100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:26.847772 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:26.848206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:26.848525 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] PATH_INFO: `/v3/users/362c9656f9b14fed96b930aa599cd480` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:26.849708 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=362c9656f9b14fed96b930aa599cd480)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:26.851792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:26.938244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '362c9656f9b14fed96b930aa599cd480'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:26.939093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '362c9656f9b14fed96b930aa599cd480'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:26.940666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '362c9656f9b14fed96b930aa599cd480'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:26.950729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '362c9656f9b14fed96b930aa599cd480'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:26.969398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '362c9656f9b14fed96b930aa599cd480'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:26.981893 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:27.005656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1345/2680] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:19:26 2022] DELETE /identity/v3/users/362c9656f9b14fed96b930aa599cd480 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:27.010072 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7792a64-9e33-4537-b8df-96ca4042d52a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:27.010498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7792a64-9e33-4537-b8df-96ca4042d52a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:27.010617 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7792a64-9e33-4537-b8df-96ca4042d52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:27.036156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b7792a64-9e33-4537-b8df-96ca4042d52a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:27.037505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b7792a64-9e33-4537-b8df-96ca4042d52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:27.127266 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1336/2681] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:27.134076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:27.135820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:27.241978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f3fdf-d49f-4fe9-9535-b859eee0dc8a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:27.248887 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37111b61-49d5-4d72-b928-e0641290128c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:27.249584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37111b61-49d5-4d72-b928-e0641290128c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:27.250059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37111b61-49d5-4d72-b928-e0641290128c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:27.250347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37111b61-49d5-4d72-b928-e0641290128c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:27.268334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37111b61-49d5-4d72-b928-e0641290128c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:27.270792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37111b61-49d5-4d72-b928-e0641290128c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:27.286741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1346/2682] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:27.665314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7792a64-9e33-4537-b8df-96ca4042d52a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:27.667117 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b7792a64-9e33-4537-b8df-96ca4042d52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:27.692800 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7792a64-9e33-4537-b8df-96ca4042d52a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:27.733335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:27.734504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:27.736259 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:27.736886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] PATH_INFO: `/v3/projects/c2e9ecda937c4259bf440a2fe303d5c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:27.740889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2e9ecda937c4259bf440a2fe303d5c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:27.747112 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:27.862516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2e9ecda937c4259bf440a2fe303d5c5', 'request_id': 'req-e87cd1e4-d192-465c-b7f0-54055885a94a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:27.875654 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2e9ecda937c4259bf440a2fe303d5c5', 'request_id': 'req-e87cd1e4-d192-465c-b7f0-54055885a94a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:27.883268 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2e9ecda937c4259bf440a2fe303d5c5', 'request_id': 'req-e87cd1e4-d192-465c-b7f0-54055885a94a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:27.909236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] The token cache is being invalidate because project c2e9ecda937c4259bf440a2fe303d5c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:19:27.909620 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:27.910407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e87cd1e4-d192-465c-b7f0-54055885a94a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:27.912609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1337/2683] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:19:27 2022] DELETE /identity/v3/projects/c2e9ecda937c4259bf440a2fe303d5c5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:27.919079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c000ab-9b29-4a58-be32-1c42d6f6ee95 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:27.919596 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c000ab-9b29-4a58-be32-1c42d6f6ee95 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:27.920053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c000ab-9b29-4a58-be32-1c42d6f6ee95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:27.949404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f2c000ab-9b29-4a58-be32-1c42d6f6ee95 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:27.950719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f2c000ab-9b29-4a58-be32-1c42d6f6ee95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:28.066912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1347/2684] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:28.076358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c01305-f3bd-41d4-9845-8b9f87f22b1b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:28.076701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c01305-f3bd-41d4-9845-8b9f87f22b1b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:28.076980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c01305-f3bd-41d4-9845-8b9f87f22b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:28.107102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c0c01305-f3bd-41d4-9845-8b9f87f22b1b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:28.108503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c0c01305-f3bd-41d4-9845-8b9f87f22b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:28.123025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1338/2685] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:28.125254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c000ab-9b29-4a58-be32-1c42d6f6ee95 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:28.141500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c000ab-9b29-4a58-be32-1c42d6f6ee95 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:28.150324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04224e-64de-473f-938e-b5239f8b56bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:28.151381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04224e-64de-473f-938e-b5239f8b56bb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:28.151867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04224e-64de-473f-938e-b5239f8b56bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:28.159046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04224e-64de-473f-938e-b5239f8b56bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:28.160016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04224e-64de-473f-938e-b5239f8b56bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:19:28.160332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04224e-64de-473f-938e-b5239f8b56bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:28.162070 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04224e-64de-473f-938e-b5239f8b56bb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:28.182178 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1348/2686] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:19:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:19:28.187704 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c01305-f3bd-41d4-9845-8b9f87f22b1b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:28.200482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c01305-f3bd-41d4-9845-8b9f87f22b1b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:28.206134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bafc7-4c71-440d-b38f-c8938827fec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:28.207059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bafc7-4c71-440d-b38f-c8938827fec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:28.207691 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bafc7-4c71-440d-b38f-c8938827fec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:28.208277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bafc7-4c71-440d-b38f-c8938827fec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:28.209300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bafc7-4c71-440d-b38f-c8938827fec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:28.211840 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bafc7-4c71-440d-b38f-c8938827fec0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:28.235853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:28.261201 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1339/2687] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:28 2022] POST /identity/v3/projects => generated 387 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:28.263345 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04224e-64de-473f-938e-b5239f8b56bb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:28.274050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04224e-64de-473f-938e-b5239f8b56bb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:28.387392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fba06-9221-419d-8b98-281bedcc90e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:28.388684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fba06-9221-419d-8b98-281bedcc90e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:28.389146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fba06-9221-419d-8b98-281bedcc90e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:28.389574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fba06-9221-419d-8b98-281bedcc90e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:28.390435 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fba06-9221-419d-8b98-281bedcc90e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:28.393914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1fba06-9221-419d-8b98-281bedcc90e2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:28.446555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bafc7-4c71-440d-b38f-c8938827fec0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:28.446765 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1349/2688] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:28 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:28.467668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bafc7-4c71-440d-b38f-c8938827fec0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:28.476435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174bf29-1661-4f36-8bcd-b36d1c173351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:28.477647 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174bf29-1661-4f36-8bcd-b36d1c173351 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:28.478058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174bf29-1661-4f36-8bcd-b36d1c173351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:28.478408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174bf29-1661-4f36-8bcd-b36d1c173351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:28.479137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174bf29-1661-4f36-8bcd-b36d1c173351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:28.481621 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174bf29-1661-4f36-8bcd-b36d1c173351 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:28.497977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1340/2689] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:28.501938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fba06-9221-419d-8b98-281bedcc90e2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:28.517371 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fba06-9221-419d-8b98-281bedcc90e2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:28.523413 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcf2f2a-88cb-4ee9-9cd9-5946812c492e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:28.524160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf2f2a-88cb-4ee9-9cd9-5946812c492e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:28.524516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf2f2a-88cb-4ee9-9cd9-5946812c492e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:28.524895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcf2f2a-88cb-4ee9-9cd9-5946812c492e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:28.525933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf2f2a-88cb-4ee9-9cd9-5946812c492e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:28.528646 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcf2f2a-88cb-4ee9-9cd9-5946812c492e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:28.551180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1350/2690] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:28.555717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174bf29-1661-4f36-8bcd-b36d1c173351 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:28.570288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174bf29-1661-4f36-8bcd-b36d1c173351 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:28.581129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f19978-a372-4323-af5a-2f1230d6bf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:28.584216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f19978-a372-4323-af5a-2f1230d6bf04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:28.584500 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f19978-a372-4323-af5a-2f1230d6bf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:28.585407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f19978-a372-4323-af5a-2f1230d6bf04 admin admin] PATH_INFO: `/v3/projects/83af8ee1f6fe45edba552920afb8e705/users/1600d7d176eb402aa408d6de9cf4ce45/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:28.617338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f19978-a372-4323-af5a-2f1230d6bf04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83af8ee1f6fe45edba552920afb8e705, user_id=1600d7d176eb402aa408d6de9cf4ce45, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:28.620669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f19978-a372-4323-af5a-2f1230d6bf04 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:28.658794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1341/2691] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:28 2022] PUT /identity/v3/projects/83af8ee1f6fe45edba552920afb8e705/users/1600d7d176eb402aa408d6de9cf4ce45/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:28.665874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af0c34-ed55-47ba-b27f-e13f8a505455 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:28.666164 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af0c34-ed55-47ba-b27f-e13f8a505455 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:28.666444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af0c34-ed55-47ba-b27f-e13f8a505455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:28.682862 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:28.704491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-47af0c34-ed55-47ba-b27f-e13f8a505455 None None] MFA Rules not processed for user `1600d7d176eb402aa408d6de9cf4ce45`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:28.706389 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-47af0c34-ed55-47ba-b27f-e13f8a505455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:28.823623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1351/2692] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:28.828837 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f19978-a372-4323-af5a-2f1230d6bf04 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:28.936274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f19978-a372-4323-af5a-2f1230d6bf04 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:28.942048 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d388ef6-13e2-48ca-8dc1-a176250f5bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:28.943564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d388ef6-13e2-48ca-8dc1-a176250f5bbf admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:28.943564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d388ef6-13e2-48ca-8dc1-a176250f5bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:28.943564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d388ef6-13e2-48ca-8dc1-a176250f5bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:28.944574 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d388ef6-13e2-48ca-8dc1-a176250f5bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:28.946828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d388ef6-13e2-48ca-8dc1-a176250f5bbf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:28.990732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1342/2693] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:28 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:28.992256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af0c34-ed55-47ba-b27f-e13f8a505455 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:29.006560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af0c34-ed55-47ba-b27f-e13f8a505455 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:29.096404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d367c01-36a7-4423-a116-9be9cab31db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:29.097841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d367c01-36a7-4423-a116-9be9cab31db1 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:29.098299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d367c01-36a7-4423-a116-9be9cab31db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:29.098749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d367c01-36a7-4423-a116-9be9cab31db1 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:29.100574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d367c01-36a7-4423-a116-9be9cab31db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:29.103286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d367c01-36a7-4423-a116-9be9cab31db1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:29.142381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1352/2694] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:28 2022] POST /identity/v3/users => generated 300 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:29.142780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d388ef6-13e2-48ca-8dc1-a176250f5bbf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:29.160841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d388ef6-13e2-48ca-8dc1-a176250f5bbf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:29.173974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7b683-1509-4967-9808-0540f39b8001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:29.175184 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b683-1509-4967-9808-0540f39b8001 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:29.175666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b683-1509-4967-9808-0540f39b8001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:29.176137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f7b683-1509-4967-9808-0540f39b8001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:29.177029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7b683-1509-4967-9808-0540f39b8001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:29.180003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f7b683-1509-4967-9808-0540f39b8001 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:29.214052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d367c01-36a7-4423-a116-9be9cab31db1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:29.242692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1343/2695] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:29.262516 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d367c01-36a7-4423-a116-9be9cab31db1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:29.285624 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49e70d-82f6-413b-a077-4f07d9dbca6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:29.288388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e70d-82f6-413b-a077-4f07d9dbca6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:29.288388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e70d-82f6-413b-a077-4f07d9dbca6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:29.288388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd49e70d-82f6-413b-a077-4f07d9dbca6c admin admin] PATH_INFO: `/v3/projects/22f3f8d182794cc0b0df2726973d0f31/users/86e3869ff137482b8d42f414ed696ce4/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:29.378414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49e70d-82f6-413b-a077-4f07d9dbca6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f3f8d182794cc0b0df2726973d0f31, user_id=86e3869ff137482b8d42f414ed696ce4, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:29.392787 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd49e70d-82f6-413b-a077-4f07d9dbca6c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:29.425337 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1353/2696] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:29 2022] PUT /identity/v3/projects/22f3f8d182794cc0b0df2726973d0f31/users/86e3869ff137482b8d42f414ed696ce4/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:29.429035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7b683-1509-4967-9808-0540f39b8001 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:29.445402 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f7b683-1509-4967-9808-0540f39b8001 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:29.578633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbe6bb-3a7e-4699-bf70-c2968ea2b5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:29.583404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbe6bb-3a7e-4699-bf70-c2968ea2b5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:29.583949 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbe6bb-3a7e-4699-bf70-c2968ea2b5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:29.584451 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbe6bb-3a7e-4699-bf70-c2968ea2b5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:29.585277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbe6bb-3a7e-4699-bf70-c2968ea2b5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:29.587644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbe6bb-3a7e-4699-bf70-c2968ea2b5d8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:29.609580 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1344/2697] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:29.614375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49e70d-82f6-413b-a077-4f07d9dbca6c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:29.630355 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd49e70d-82f6-413b-a077-4f07d9dbca6c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:29.635661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f041955-c3e3-45b4-b2d0-0118dfb9a09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:29.636336 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f041955-c3e3-45b4-b2d0-0118dfb9a09d admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:29.637178 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f041955-c3e3-45b4-b2d0-0118dfb9a09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:29.637428 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f041955-c3e3-45b4-b2d0-0118dfb9a09d admin admin] PATH_INFO: `/v3/projects/22f3f8d182794cc0b0df2726973d0f31/users/86e3869ff137482b8d42f414ed696ce4/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:29.639888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f041955-c3e3-45b4-b2d0-0118dfb9a09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f3f8d182794cc0b0df2726973d0f31, user_id=86e3869ff137482b8d42f414ed696ce4, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:29.642954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f041955-c3e3-45b4-b2d0-0118dfb9a09d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:29.666658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1354/2698] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:29 2022] PUT /identity/v3/projects/22f3f8d182794cc0b0df2726973d0f31/users/86e3869ff137482b8d42f414ed696ce4/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:29.677612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3716bc-5fe3-47d3-be0b-6e3fd7f82bec None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:29.677856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3716bc-5fe3-47d3-be0b-6e3fd7f82bec None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:29.678044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3716bc-5fe3-47d3-be0b-6e3fd7f82bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:29.756321 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fd3716bc-5fe3-47d3-be0b-6e3fd7f82bec None None] MFA Rules not processed for user `86e3869ff137482b8d42f414ed696ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:29.757724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fd3716bc-5fe3-47d3-be0b-6e3fd7f82bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:29.800935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:29.924024 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1345/2699] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:29.939733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f041955-c3e3-45b4-b2d0-0118dfb9a09d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:29.942139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4f041955-c3e3-45b4-b2d0-0118dfb9a09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:30.036344 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:30.041478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f041955-c3e3-45b4-b2d0-0118dfb9a09d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:30.048761 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c4d48-a237-43ee-ab51-121bfc7cf443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:30.050013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c4d48-a237-43ee-ab51-121bfc7cf443 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:30.050499 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c4d48-a237-43ee-ab51-121bfc7cf443 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:30.051013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c4d48-a237-43ee-ab51-121bfc7cf443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:30.067675 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c4d48-a237-43ee-ab51-121bfc7cf443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:30.069755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c4d48-a237-43ee-ab51-121bfc7cf443 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:30.087635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1355/2700] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:30.314203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3716bc-5fe3-47d3-be0b-6e3fd7f82bec None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:30.396561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3716bc-5fe3-47d3-be0b-6e3fd7f82bec None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:30.404138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:30.405023 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:30.405361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:30.405650 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] PATH_INFO: `/v3/users/1600d7d176eb402aa408d6de9cf4ce45` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:30.407012 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1600d7d176eb402aa408d6de9cf4ce45)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:30.409422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:30.502211 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1600d7d176eb402aa408d6de9cf4ce45'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:30.532665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1600d7d176eb402aa408d6de9cf4ce45'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:30.534537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1600d7d176eb402aa408d6de9cf4ce45'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:30.536223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1600d7d176eb402aa408d6de9cf4ce45'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:30.558861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1600d7d176eb402aa408d6de9cf4ce45'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:30.630824 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1346/2701] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:19:30 2022] DELETE /identity/v3/users/1600d7d176eb402aa408d6de9cf4ce45 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:30.635374 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9cc1b-04e7-42fc-bebe-7c6d10e9a032 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:30.635721 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9cc1b-04e7-42fc-bebe-7c6d10e9a032 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:30.636152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9cc1b-04e7-42fc-bebe-7c6d10e9a032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:30.662144 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-32f9cc1b-04e7-42fc-bebe-7c6d10e9a032 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:30.663508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-32f9cc1b-04e7-42fc-bebe-7c6d10e9a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:30.745415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1356/2702] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:30.758692 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:30.760996 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:30.841411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:30.876308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cc438e-fa3c-4117-a884-cad93d765bfb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:30.885591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80931572-fe5c-4a7b-a5c8-cb429d7a8e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:30.887792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80931572-fe5c-4a7b-a5c8-cb429d7a8e68 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:30.888280 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80931572-fe5c-4a7b-a5c8-cb429d7a8e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:30.888793 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80931572-fe5c-4a7b-a5c8-cb429d7a8e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:30.903712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80931572-fe5c-4a7b-a5c8-cb429d7a8e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:30.905640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80931572-fe5c-4a7b-a5c8-cb429d7a8e68 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:30.923304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1347/2703] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:31.540855 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9cc1b-04e7-42fc-bebe-7c6d10e9a032 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:31.544553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-32f9cc1b-04e7-42fc-bebe-7c6d10e9a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:31.559324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9cc1b-04e7-42fc-bebe-7c6d10e9a032 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:31.566969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:31.568398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:31.568398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:31.568830 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] PATH_INFO: `/v3/projects/83af8ee1f6fe45edba552920afb8e705` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:31.570528 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] RBAC: Authorizing `identity:delete_project(project_id=83af8ee1f6fe45edba552920afb8e705)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:31.573361 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:31.629792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbb7ed-90c1-4ca4-8ad3-768095756d8b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:31.630301 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbb7ed-90c1-4ca4-8ad3-768095756d8b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:31.630411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbb7ed-90c1-4ca4-8ad3-768095756d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:31.650973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83af8ee1f6fe45edba552920afb8e705', 'request_id': 'req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:31.669233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83af8ee1f6fe45edba552920afb8e705', 'request_id': 'req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:31.686769 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7afbb7ed-90c1-4ca4-8ad3-768095756d8b None None] MFA Rules not processed for user `4320be012bb045afb47948c6c7217a70`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:31.688051 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7afbb7ed-90c1-4ca4-8ad3-768095756d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:31.693121 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83af8ee1f6fe45edba552920afb8e705', 'request_id': 'req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:31.709229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] The token cache is being invalidate because project 83af8ee1f6fe45edba552920afb8e705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:19:31.709756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:31.712362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:31.715545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1357/2704] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:19:31 2022] DELETE /identity/v3/projects/83af8ee1f6fe45edba552920afb8e705 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:31.719762 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:31.721311 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:31.807583 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1348/2705] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:31.822727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbb7ed-90c1-4ca4-8ad3-768095756d8b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:31.825754 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7afbb7ed-90c1-4ca4-8ad3-768095756d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:31.844842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cebb8b-268a-4ed1-a0ba-2d0dc10ee497 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:31.851079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:31.851731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:31.852434 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:31.852765 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] PATH_INFO: `/v3/users/86e3869ff137482b8d42f414ed696ce4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:31.853977 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86e3869ff137482b8d42f414ed696ce4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:31.856922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:31.878659 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:31.930539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86e3869ff137482b8d42f414ed696ce4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:31.931313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86e3869ff137482b8d42f414ed696ce4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:31.932013 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86e3869ff137482b8d42f414ed696ce4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:31.947690 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86e3869ff137482b8d42f414ed696ce4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:31.987927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86e3869ff137482b8d42f414ed696ce4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:32.020171 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbb7ed-90c1-4ca4-8ad3-768095756d8b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:32.021812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1358/2706] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:19:31 2022] DELETE /identity/v3/users/86e3869ff137482b8d42f414ed696ce4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:19:32.118287 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:32.138286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4778b-4d69-42a8-bf1c-f2a669847b0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:32.139097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4778b-4d69-42a8-bf1c-f2a669847b0a service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:32.139481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4778b-4d69-42a8-bf1c-f2a669847b0a service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:32.139815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4778b-4d69-42a8-bf1c-f2a669847b0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:32.142103 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e3e4778b-4d69-42a8-bf1c-f2a669847b0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:32.226073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4778b-4d69-42a8-bf1c-f2a669847b0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:32.227977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4778b-4d69-42a8-bf1c-f2a669847b0a service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:32.239870 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1349/2707] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:32.421448 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:32.423161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:32.528841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111606e-6080-4f9e-8d28-f7a10c7fa9e6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:32.538951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:32.539605 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:32.539879 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:32.540151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] PATH_INFO: `/v3/projects/22f3f8d182794cc0b0df2726973d0f31` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:32.541314 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f3f8d182794cc0b0df2726973d0f31)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:32.544071 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:32.622895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f3f8d182794cc0b0df2726973d0f31', 'request_id': 'req-82b01d62-eb62-41f9-98a8-d5bcae86a69a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:32.632680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f3f8d182794cc0b0df2726973d0f31', 'request_id': 'req-82b01d62-eb62-41f9-98a8-d5bcae86a69a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:32.641801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f3f8d182794cc0b0df2726973d0f31', 'request_id': 'req-82b01d62-eb62-41f9-98a8-d5bcae86a69a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:32.654589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] The token cache is being invalidate because project 22f3f8d182794cc0b0df2726973d0f31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:19:32.654877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:32.655684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-82b01d62-eb62-41f9-98a8-d5bcae86a69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:32.657319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1359/2708] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:19:32 2022] DELETE /identity/v3/projects/22f3f8d182794cc0b0df2726973d0f31 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:32.663897 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399ba95-ffe3-48a8-8ba6-090031e4cef2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:32.664419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399ba95-ffe3-48a8-8ba6-090031e4cef2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:32.664419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399ba95-ffe3-48a8-8ba6-090031e4cef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:32.698025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d399ba95-ffe3-48a8-8ba6-090031e4cef2 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:32.699019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d399ba95-ffe3-48a8-8ba6-090031e4cef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:32.782097 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1350/2709] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:32.789421 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64f5c7-c40d-482a-88b9-a52b4ad1c283 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:32.789702 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64f5c7-c40d-482a-88b9-a52b4ad1c283 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:32.790004 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64f5c7-c40d-482a-88b9-a52b4ad1c283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:32.813753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4c64f5c7-c40d-482a-88b9-a52b4ad1c283 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:32.815036 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4c64f5c7-c40d-482a-88b9-a52b4ad1c283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:32.824763 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1360/2710] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:32.829218 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399ba95-ffe3-48a8-8ba6-090031e4cef2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:32.844628 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399ba95-ffe3-48a8-8ba6-090031e4cef2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:32.850642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb2f29-28e8-4310-b8c7-9a217a3d9a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:32.851273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb2f29-28e8-4310-b8c7-9a217a3d9a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:32.851551 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb2f29-28e8-4310-b8c7-9a217a3d9a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:32.851830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb2f29-28e8-4310-b8c7-9a217a3d9a9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:32.852513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb2f29-28e8-4310-b8c7-9a217a3d9a9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:19:32.852816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb2f29-28e8-4310-b8c7-9a217a3d9a9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:32.854497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb2f29-28e8-4310-b8c7-9a217a3d9a9e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:32.869335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1351/2711] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:19:32.875777 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64f5c7-c40d-482a-88b9-a52b4ad1c283 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:32.888196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64f5c7-c40d-482a-88b9-a52b4ad1c283 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:32.893949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40129af8-e110-4617-8b14-5724936549e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:32.894737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40129af8-e110-4617-8b14-5724936549e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:32.895187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40129af8-e110-4617-8b14-5724936549e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:32.895470 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40129af8-e110-4617-8b14-5724936549e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:32.896132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40129af8-e110-4617-8b14-5724936549e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:32.899155 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40129af8-e110-4617-8b14-5724936549e3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:32.922124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:32.939508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1361/2712] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:32 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:32.944419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb2f29-28e8-4310-b8c7-9a217a3d9a9e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:32.958297 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb2f29-28e8-4310-b8c7-9a217a3d9a9e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:33.044816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1315a-f38e-44b3-aa2c-d647158ac3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:33.045446 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1315a-f38e-44b3-aa2c-d647158ac3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:33.045726 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1315a-f38e-44b3-aa2c-d647158ac3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:33.046183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1315a-f38e-44b3-aa2c-d647158ac3f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:33.046858 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1315a-f38e-44b3-aa2c-d647158ac3f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:33.049223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1315a-f38e-44b3-aa2c-d647158ac3f9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:33.079701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1352/2713] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:32 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:19:33.081297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40129af8-e110-4617-8b14-5724936549e3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:33.093206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40129af8-e110-4617-8b14-5724936549e3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:33.098741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1435df-c84b-42eb-ba09-c05eda92ebe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:33.099366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1435df-c84b-42eb-ba09-c05eda92ebe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:33.099647 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1435df-c84b-42eb-ba09-c05eda92ebe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:33.099922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1435df-c84b-42eb-ba09-c05eda92ebe9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:33.100580 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1435df-c84b-42eb-ba09-c05eda92ebe9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:33.102276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1435df-c84b-42eb-ba09-c05eda92ebe9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:33.114554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1362/2714] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:33.118492 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1315a-f38e-44b3-aa2c-d647158ac3f9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:33.130489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1315a-f38e-44b3-aa2c-d647158ac3f9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:33.143337 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3e257-e644-40d3-baec-cd11936ea771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:33.143564 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3e257-e644-40d3-baec-cd11936ea771 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:33.143666 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3e257-e644-40d3-baec-cd11936ea771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:33.143914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3e257-e644-40d3-baec-cd11936ea771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:33.144640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3e257-e644-40d3-baec-cd11936ea771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:33.146986 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b3e257-e644-40d3-baec-cd11936ea771 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:33.162332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1353/2715] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:33.167120 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1435df-c84b-42eb-ba09-c05eda92ebe9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:33.178346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1435df-c84b-42eb-ba09-c05eda92ebe9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:33.185248 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1320df-ef32-4e86-b995-9167a9081857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:33.186467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1320df-ef32-4e86-b995-9167a9081857 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:33.186467 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1320df-ef32-4e86-b995-9167a9081857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:33.186892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1320df-ef32-4e86-b995-9167a9081857 admin admin] PATH_INFO: `/v3/projects/632961f7d29141729ae00080ed7d4076/users/b00f899e2ea54ede97dc847bf332db02/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:33.208003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1320df-ef32-4e86-b995-9167a9081857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=632961f7d29141729ae00080ed7d4076, user_id=b00f899e2ea54ede97dc847bf332db02, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:33.210756 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1320df-ef32-4e86-b995-9167a9081857 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:33.229283 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1363/2716] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:33 2022] PUT /identity/v3/projects/632961f7d29141729ae00080ed7d4076/users/b00f899e2ea54ede97dc847bf332db02/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:33.235509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf0caa-1aa4-402d-8cdb-50130d310121 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:33.235509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf0caa-1aa4-402d-8cdb-50130d310121 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:33.235509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cf0caa-1aa4-402d-8cdb-50130d310121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:33.245221 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:33.258540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-35cf0caa-1aa4-402d-8cdb-50130d310121 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:33.262180 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-35cf0caa-1aa4-402d-8cdb-50130d310121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:33.340368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1354/2717] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:33.347498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1320df-ef32-4e86-b995-9167a9081857 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:33.350812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7d1320df-ef32-4e86-b995-9167a9081857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:33.420586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1320df-ef32-4e86-b995-9167a9081857 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:33.425298 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e218bd6-36e7-4304-943b-7e643b067292 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:33.425865 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e218bd6-36e7-4304-943b-7e643b067292 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:33.426254 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e218bd6-36e7-4304-943b-7e643b067292 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:33.426737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e218bd6-36e7-4304-943b-7e643b067292 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:33.437433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e218bd6-36e7-4304-943b-7e643b067292 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:33.438956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e218bd6-36e7-4304-943b-7e643b067292 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:33.449269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1364/2718] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:19:34.379727 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ca735-8914-4c75-9912-7487a549475c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:34.380452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ca735-8914-4c75-9912-7487a549475c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:34.380452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098ca735-8914-4c75-9912-7487a549475c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:34.416368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-098ca735-8914-4c75-9912-7487a549475c None None] MFA Rules not processed for user `b00f899e2ea54ede97dc847bf332db02`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:34.419299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-098ca735-8914-4c75-9912-7487a549475c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:34.502913 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e218bd6-36e7-4304-943b-7e643b067292 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:34.506937 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0e218bd6-36e7-4304-943b-7e643b067292 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:34.531429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1355/2719] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:34.535860 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ca735-8914-4c75-9912-7487a549475c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:34.558453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098ca735-8914-4c75-9912-7487a549475c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:34.574913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed93e14-f472-45f9-ad1f-f97a4712e7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:34.575871 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed93e14-f472-45f9-ad1f-f97a4712e7ac admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:34.576248 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed93e14-f472-45f9-ad1f-f97a4712e7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:34.576643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed93e14-f472-45f9-ad1f-f97a4712e7ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:34.578420 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed93e14-f472-45f9-ad1f-f97a4712e7ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:34.581044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed93e14-f472-45f9-ad1f-f97a4712e7ac admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:34.607322 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e218bd6-36e7-4304-943b-7e643b067292 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:34.620054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f42489-6229-47b4-aa0e-ad4df3de5644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:34.621081 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f42489-6229-47b4-aa0e-ad4df3de5644 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:34.621949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f42489-6229-47b4-aa0e-ad4df3de5644 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:34.622091 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f42489-6229-47b4-aa0e-ad4df3de5644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:34.628578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-96f42489-6229-47b4-aa0e-ad4df3de5644 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:34.637998 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1356/2720] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:34 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:34.642852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed93e14-f472-45f9-ad1f-f97a4712e7ac admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:34.732750 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed93e14-f472-45f9-ad1f-f97a4712e7ac admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:34.739612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73ce31-97fa-4efe-974a-8c202e8f8916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:34.744006 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73ce31-97fa-4efe-974a-8c202e8f8916 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:34.745883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73ce31-97fa-4efe-974a-8c202e8f8916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:34.746820 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73ce31-97fa-4efe-974a-8c202e8f8916 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:34.747324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f42489-6229-47b4-aa0e-ad4df3de5644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:34.747702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73ce31-97fa-4efe-974a-8c202e8f8916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:34.749326 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f42489-6229-47b4-aa0e-ad4df3de5644 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:34.752908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73ce31-97fa-4efe-974a-8c202e8f8916 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:34.755135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:34.779478 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1365/2721] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:34.781866 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1357/2722] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:34 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:34.784451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f42489-6229-47b4-aa0e-ad4df3de5644 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:34.796114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f42489-6229-47b4-aa0e-ad4df3de5644 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:34.801102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fc40c-9d0f-4634-9631-eb591726b84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:34.802046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fc40c-9d0f-4634-9631-eb591726b84b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:34.802046 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fc40c-9d0f-4634-9631-eb591726b84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:34.802307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586fc40c-9d0f-4634-9631-eb591726b84b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:34.802981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fc40c-9d0f-4634-9631-eb591726b84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:34.804566 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586fc40c-9d0f-4634-9631-eb591726b84b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:34.821810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1366/2723] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:34.823591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73ce31-97fa-4efe-974a-8c202e8f8916 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:34.834062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:34.850208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73ce31-97fa-4efe-974a-8c202e8f8916 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:34.860821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e43139-6b9a-4e7b-8b99-26150fbd7714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:34.861701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e43139-6b9a-4e7b-8b99-26150fbd7714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:34.862082 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e43139-6b9a-4e7b-8b99-26150fbd7714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:34.862368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e43139-6b9a-4e7b-8b99-26150fbd7714 admin admin] PATH_INFO: `/v3/projects/5a1bd97db71c4dafb733d670661a632d/users/dc6df949d8114b0e8e3d1ae1a14f12f8/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:34.897552 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e43139-6b9a-4e7b-8b99-26150fbd7714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1bd97db71c4dafb733d670661a632d, user_id=dc6df949d8114b0e8e3d1ae1a14f12f8, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:34.900404 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e43139-6b9a-4e7b-8b99-26150fbd7714 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:34.917365 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1358/2724] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:34 2022] PUT /identity/v3/projects/5a1bd97db71c4dafb733d670661a632d/users/dc6df949d8114b0e8e3d1ae1a14f12f8/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:34.922181 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fc40c-9d0f-4634-9631-eb591726b84b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:34.932808 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586fc40c-9d0f-4634-9631-eb591726b84b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:35.043306 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e73f9-083f-4c4e-b2e6-bc2606a05f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:35.044207 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e73f9-083f-4c4e-b2e6-bc2606a05f9b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:35.046333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e73f9-083f-4c4e-b2e6-bc2606a05f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:35.046484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e73f9-083f-4c4e-b2e6-bc2606a05f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:35.047105 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e73f9-083f-4c4e-b2e6-bc2606a05f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:35.048747 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e73f9-083f-4c4e-b2e6-bc2606a05f9b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:35.072271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1367/2725] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:35.078711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e43139-6b9a-4e7b-8b99-26150fbd7714 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:35.104972 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e43139-6b9a-4e7b-8b99-26150fbd7714 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:35.114368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693fa93-c3ec-4afb-8ca2-1f4369e9b81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:35.118123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693fa93-c3ec-4afb-8ca2-1f4369e9b81e admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:35.118443 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693fa93-c3ec-4afb-8ca2-1f4369e9b81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:35.118968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693fa93-c3ec-4afb-8ca2-1f4369e9b81e admin admin] PATH_INFO: `/v3/projects/5a1bd97db71c4dafb733d670661a632d/users/dc6df949d8114b0e8e3d1ae1a14f12f8/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:35.123926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693fa93-c3ec-4afb-8ca2-1f4369e9b81e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1bd97db71c4dafb733d670661a632d, user_id=dc6df949d8114b0e8e3d1ae1a14f12f8, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:35.126740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693fa93-c3ec-4afb-8ca2-1f4369e9b81e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:35.143952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1359/2726] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:35 2022] PUT /identity/v3/projects/5a1bd97db71c4dafb733d670661a632d/users/dc6df949d8114b0e8e3d1ae1a14f12f8/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:35.963772 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7f822-ea60-424a-8777-ab3ed6ad807f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:35.964297 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7f822-ea60-424a-8777-ab3ed6ad807f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:35.965264 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7f822-ea60-424a-8777-ab3ed6ad807f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:35.999045 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-01d7f822-ea60-424a-8777-ab3ed6ad807f None None] MFA Rules not processed for user `dc6df949d8114b0e8e3d1ae1a14f12f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:36.000517 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01d7f822-ea60-424a-8777-ab3ed6ad807f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:36.118870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1368/2727] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:36.135230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693fa93-c3ec-4afb-8ca2-1f4369e9b81e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:36.139276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2693fa93-c3ec-4afb-8ca2-1f4369e9b81e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:36.247077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693fa93-c3ec-4afb-8ca2-1f4369e9b81e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:36.251989 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a31d3-ec92-41de-92bf-0beeee760f71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:36.272499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a31d3-ec92-41de-92bf-0beeee760f71 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:36.272814 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a31d3-ec92-41de-92bf-0beeee760f71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:36.273313 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a31d3-ec92-41de-92bf-0beeee760f71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:36.361264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a31d3-ec92-41de-92bf-0beeee760f71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:36.363019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a31d3-ec92-41de-92bf-0beeee760f71 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:36.382877 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1360/2728] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:38.002414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7f822-ea60-424a-8777-ab3ed6ad807f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:38.014461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7f822-ea60-424a-8777-ab3ed6ad807f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:38.019594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a20d7-a5d9-4e75-9367-f2b1de17a85c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:38.020583 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a20d7-a5d9-4e75-9367-f2b1de17a85c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:38.021034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a20d7-a5d9-4e75-9367-f2b1de17a85c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:38.021475 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a20d7-a5d9-4e75-9367-f2b1de17a85c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:38.040955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a20d7-a5d9-4e75-9367-f2b1de17a85c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:38.043171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a20d7-a5d9-4e75-9367-f2b1de17a85c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:38.065871 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1369/2729] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:42.165083 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a31d3-ec92-41de-92bf-0beeee760f71 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:42.166734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1c8a31d3-ec92-41de-92bf-0beeee760f71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:42.250240 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a31d3-ec92-41de-92bf-0beeee760f71 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:42.256991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc95a72-461a-49fb-9123-90829a60f976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:42.258065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc95a72-461a-49fb-9123-90829a60f976 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:42.258854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc95a72-461a-49fb-9123-90829a60f976 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:42.259586 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc95a72-461a-49fb-9123-90829a60f976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:42.266021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-acc95a72-461a-49fb-9123-90829a60f976 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:42.337626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:42.378849 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc95a72-461a-49fb-9123-90829a60f976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:42.380976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc95a72-461a-49fb-9123-90829a60f976 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:42.394507 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1361/2730] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:45.334992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900844f-50d3-44d4-9fab-4a5985988c78 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:45.335600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900844f-50d3-44d4-9fab-4a5985988c78 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:45.335703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900844f-50d3-44d4-9fab-4a5985988c78 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:45.337323 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1370/2731] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:19:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:19:45.344322 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af85edf-3fbf-4f80-893a-db4cdc32f988 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:45.344916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af85edf-3fbf-4f80-893a-db4cdc32f988 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:45.344916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af85edf-3fbf-4f80-893a-db4cdc32f988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:45.404269 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4af85edf-3fbf-4f80-893a-db4cdc32f988 None None] MFA Rules not processed for user `b37a4275e7c6441aa9e8bb0298f56589`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:45.405599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4af85edf-3fbf-4f80-893a-db4cdc32f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:45.413642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1362/2732] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:19:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:45.432232 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900844f-50d3-44d4-9fab-4a5985988c78 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:45.462445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900844f-50d3-44d4-9fab-4a5985988c78 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:45.467354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ff608-a481-4067-b04f-ca305cf9220c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:45.468204 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ff608-a481-4067-b04f-ca305cf9220c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:45.468643 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ff608-a481-4067-b04f-ca305cf9220c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:45.469073 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24ff608-a481-4067-b04f-ca305cf9220c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:45.477923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ff608-a481-4067-b04f-ca305cf9220c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:45.481511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24ff608-a481-4067-b04f-ca305cf9220c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:45.492804 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1371/2733] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:45.970601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af85edf-3fbf-4f80-893a-db4cdc32f988 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:45.972340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4af85edf-3fbf-4f80-893a-db4cdc32f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:45.990226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af85edf-3fbf-4f80-893a-db4cdc32f988 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:45.994978 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:45.995603 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:45.996504 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:45.996771 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] PATH_INFO: `/v3/users/0809d96091c2476c83ec0c7c6c5a1509` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:45.998115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0809d96091c2476c83ec0c7c6c5a1509)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:46.000031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:46.051599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0809d96091c2476c83ec0c7c6c5a1509'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:46.058914 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0809d96091c2476c83ec0c7c6c5a1509'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:46.059807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0809d96091c2476c83ec0c7c6c5a1509'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:46.060469 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0809d96091c2476c83ec0c7c6c5a1509'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:46.067833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3cfab346-aede-4ead-96fe-2ea9a10d6f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0809d96091c2476c83ec0c7c6c5a1509'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:46.097190 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1363/2734] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:19:45 2022] DELETE /identity/v3/users/0809d96091c2476c83ec0c7c6c5a1509 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:46.306755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ff608-a481-4067-b04f-ca305cf9220c service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:46.322418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24ff608-a481-4067-b04f-ca305cf9220c service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:46.385293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:46.405856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:46.406497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.406810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.407086 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] PATH_INFO: `/v3/projects/a3902dce55324b7ab42df9de97bc11b4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.408235 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3902dce55324b7ab42df9de97bc11b4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:46.410205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:46.458079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3902dce55324b7ab42df9de97bc11b4', 'request_id': 'req-12f91c1f-1e7b-423a-9153-f6e335b508d6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:46.467820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3902dce55324b7ab42df9de97bc11b4', 'request_id': 'req-12f91c1f-1e7b-423a-9153-f6e335b508d6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:46.474195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3902dce55324b7ab42df9de97bc11b4', 'request_id': 'req-12f91c1f-1e7b-423a-9153-f6e335b508d6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:46.486216 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] The token cache is being invalidate because project a3902dce55324b7ab42df9de97bc11b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:19:46.486511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:46.487156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-12f91c1f-1e7b-423a-9153-f6e335b508d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:46.488639 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1372/2735] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:19:46 2022] DELETE /identity/v3/projects/a3902dce55324b7ab42df9de97bc11b4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:46.493585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fcaa8-53c2-4584-a65e-4aef13a08a71 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.494094 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fcaa8-53c2-4584-a65e-4aef13a08a71 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.494222 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4fcaa8-53c2-4584-a65e-4aef13a08a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.514558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7d4fcaa8-53c2-4584-a65e-4aef13a08a71 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:46.515776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d4fcaa8-53c2-4584-a65e-4aef13a08a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:46.575644 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1364/2736] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:46.583744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ea867-9daf-4e29-9550-718570bcaef5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.584024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ea867-9daf-4e29-9550-718570bcaef5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.584301 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35ea867-9daf-4e29-9550-718570bcaef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.606543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b35ea867-9daf-4e29-9550-718570bcaef5 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:46.608209 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b35ea867-9daf-4e29-9550-718570bcaef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:46.616347 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1373/2737] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:46.621964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fcaa8-53c2-4584-a65e-4aef13a08a71 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:46.634443 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4fcaa8-53c2-4584-a65e-4aef13a08a71 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:46.640305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0c840-c9a1-485e-8966-47c0e26bc3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:46.640865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0c840-c9a1-485e-8966-47c0e26bc3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.641147 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0c840-c9a1-485e-8966-47c0e26bc3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.641424 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0c840-c9a1-485e-8966-47c0e26bc3a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.642415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0c840-c9a1-485e-8966-47c0e26bc3a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:19:46.642773 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0c840-c9a1-485e-8966-47c0e26bc3a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:46.644605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0c840-c9a1-485e-8966-47c0e26bc3a7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:46.654349 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daf286-af8e-4b00-adbc-e391fb58d02e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.654631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daf286-af8e-4b00-adbc-e391fb58d02e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.654904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daf286-af8e-4b00-adbc-e391fb58d02e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.656449 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1374/2738] 10.222.0.16 () {54 vars in 921 bytes} [Fri Aug 19 21:19:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 19 21:19:46.660672 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1365/2739] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:19:46.662710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33190a4-47bb-4444-9a8f-13181c8a0d3e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.663106 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33190a4-47bb-4444-9a8f-13181c8a0d3e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.663450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33190a4-47bb-4444-9a8f-13181c8a0d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.665492 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0c840-c9a1-485e-8966-47c0e26bc3a7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:46.678394 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0c840-c9a1-485e-8966-47c0e26bc3a7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:46.684718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96c2d8-e5a5-47d1-b2a0-99fa5d215f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:46.684718 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96c2d8-e5a5-47d1-b2a0-99fa5d215f09 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.685030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96c2d8-e5a5-47d1-b2a0-99fa5d215f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.685237 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c96c2d8-e5a5-47d1-b2a0-99fa5d215f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.685960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96c2d8-e5a5-47d1-b2a0-99fa5d215f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:46.687856 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c96c2d8-e5a5-47d1-b2a0-99fa5d215f09 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:46.694775 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:46.708757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d33190a4-47bb-4444-9a8f-13181c8a0d3e None None] MFA Rules not processed for user `43a9ad5341b54c40a5427045660363c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:46.710003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d33190a4-47bb-4444-9a8f-13181c8a0d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:46.734369 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1366/2740] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:46 2022] POST /identity/v3/projects => generated 367 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:19:46.735724 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96c2d8-e5a5-47d1-b2a0-99fa5d215f09 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:46.801151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1375/2741] 10.222.0.16 () {58 vars in 1018 bytes} [Fri Aug 19 21:19:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:46.817444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33190a4-47bb-4444-9a8f-13181c8a0d3e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:46.818885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d33190a4-47bb-4444-9a8f-13181c8a0d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:46.820132 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c96c2d8-e5a5-47d1-b2a0-99fa5d215f09 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:46.826203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0a398-4efe-4cee-a0f1-14abd7da6dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:46.826815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0a398-4efe-4cee-a0f1-14abd7da6dca admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.827088 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0a398-4efe-4cee-a0f1-14abd7da6dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.827985 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e0a398-4efe-4cee-a0f1-14abd7da6dca admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.828640 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0a398-4efe-4cee-a0f1-14abd7da6dca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:46.830661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e0a398-4efe-4cee-a0f1-14abd7da6dca admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:46.852970 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1367/2742] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:46 2022] POST /identity/v3/users => generated 284 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:46.857559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0a398-4efe-4cee-a0f1-14abd7da6dca admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:46.862634 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:46.868809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e0a398-4efe-4cee-a0f1-14abd7da6dca admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:46.876401 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669923c-06bd-416f-a0ad-511b97f9aa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:46.877017 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669923c-06bd-416f-a0ad-511b97f9aa42 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.877298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669923c-06bd-416f-a0ad-511b97f9aa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.877599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669923c-06bd-416f-a0ad-511b97f9aa42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.878282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669923c-06bd-416f-a0ad-511b97f9aa42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:46.879890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669923c-06bd-416f-a0ad-511b97f9aa42 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:46.883955 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33190a4-47bb-4444-9a8f-13181c8a0d3e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:46.891356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1368/2743] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:46.895591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669923c-06bd-416f-a0ad-511b97f9aa42 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:46.906657 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669923c-06bd-416f-a0ad-511b97f9aa42 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:46.912235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0534b-25d0-4583-a0f8-451ae156e077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:46.912888 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0534b-25d0-4583-a0f8-451ae156e077 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.913164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0534b-25d0-4583-a0f8-451ae156e077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.913534 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0534b-25d0-4583-a0f8-451ae156e077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.914648 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0534b-25d0-4583-a0f8-451ae156e077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:46.915854 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0534b-25d0-4583-a0f8-451ae156e077 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:46.927456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1369/2744] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:46.931031 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0534b-25d0-4583-a0f8-451ae156e077 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:46.941409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0534b-25d0-4583-a0f8-451ae156e077 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:46.944527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e39b4-3d13-4903-b501-520c13abf2aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:46.945135 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e39b4-3d13-4903-b501-520c13abf2aa service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.945409 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e39b4-3d13-4903-b501-520c13abf2aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.945679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e39b4-3d13-4903-b501-520c13abf2aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.946145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf72b24-e560-4348-a471-34bc3b623bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:46.946788 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf72b24-e560-4348-a471-34bc3b623bba admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.947068 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf72b24-e560-4348-a471-34bc3b623bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.947348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf72b24-e560-4348-a471-34bc3b623bba admin admin] PATH_INFO: `/v3/projects/ec778aebbca44642b6e1ccde914af29e/users/d302c438405142c5af416787d74c41e1/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:46.966736 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf72b24-e560-4348-a471-34bc3b623bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec778aebbca44642b6e1ccde914af29e, user_id=d302c438405142c5af416787d74c41e1, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:46.969356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf72b24-e560-4348-a471-34bc3b623bba admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:46.983747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1370/2745] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:46 2022] PUT /identity/v3/projects/ec778aebbca44642b6e1ccde914af29e/users/d302c438405142c5af416787d74c41e1/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:19:46.993916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd7830-9089-4be1-ae2d-234705a69d15 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:46.994205 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd7830-9089-4be1-ae2d-234705a69d15 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:46.994481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd7830-9089-4be1-ae2d-234705a69d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:47.018390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e39b4-3d13-4903-b501-520c13abf2aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:47.019864 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e39b4-3d13-4903-b501-520c13abf2aa service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:47.025887 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d3fd7830-9089-4be1-ae2d-234705a69d15 None None] MFA Rules not processed for user `d302c438405142c5af416787d74c41e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:47.028340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d3fd7830-9089-4be1-ae2d-234705a69d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:47.034597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1376/2746] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:47.101098 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1371/2747] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:47.112563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e39b4-3d13-4903-b501-520c13abf2aa service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:47.114089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f53e39b4-3d13-4903-b501-520c13abf2aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:47.173992 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e39b4-3d13-4903-b501-520c13abf2aa service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:47.218662 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:47.223653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b509ce6-e521-4992-9a5c-a80870d51479 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:47.224261 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b509ce6-e521-4992-9a5c-a80870d51479 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:47.224540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b509ce6-e521-4992-9a5c-a80870d51479 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:47.224814 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b509ce6-e521-4992-9a5c-a80870d51479 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:47.233344 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b509ce6-e521-4992-9a5c-a80870d51479 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:47.234826 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b509ce6-e521-4992-9a5c-a80870d51479 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:47.244132 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1377/2748] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:52.807125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd7830-9089-4be1-ae2d-234705a69d15 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:52.808687 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d3fd7830-9089-4be1-ae2d-234705a69d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:52.833035 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:52.883795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd7830-9089-4be1-ae2d-234705a69d15 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:52.889304 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:52.889960 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:52.890236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:52.890506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] PATH_INFO: `/v3/users/b00f899e2ea54ede97dc847bf332db02` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:52.891635 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b00f899e2ea54ede97dc847bf332db02)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:52.893819 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:52.944412 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b00f899e2ea54ede97dc847bf332db02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:52.953411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b00f899e2ea54ede97dc847bf332db02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:52.954377 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b00f899e2ea54ede97dc847bf332db02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:52.955036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b00f899e2ea54ede97dc847bf332db02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:52.963066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b00f899e2ea54ede97dc847bf332db02'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:52.983073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1372/2749] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:19:52 2022] DELETE /identity/v3/users/b00f899e2ea54ede97dc847bf332db02 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:53.184139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b509ce6-e521-4992-9a5c-a80870d51479 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:53.198660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b509ce6-e521-4992-9a5c-a80870d51479 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:53.265450 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:53.266123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:53.266619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:53.267156 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] PATH_INFO: `/v3/projects/632961f7d29141729ae00080ed7d4076` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:53.268318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=632961f7d29141729ae00080ed7d4076)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:53.270498 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:53.322174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '632961f7d29141729ae00080ed7d4076', 'request_id': 'req-75e73949-0cb9-438a-9415-03ed07402e3e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.330353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '632961f7d29141729ae00080ed7d4076', 'request_id': 'req-75e73949-0cb9-438a-9415-03ed07402e3e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.336318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '632961f7d29141729ae00080ed7d4076', 'request_id': 'req-75e73949-0cb9-438a-9415-03ed07402e3e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.348553 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] The token cache is being invalidate because project 632961f7d29141729ae00080ed7d4076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:19:53.348839 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.349497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.351044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1378/2750] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:19:53 2022] DELETE /identity/v3/projects/632961f7d29141729ae00080ed7d4076 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:53.355058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:53.356646 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:53.370019 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ceee59-afe0-45f7-9872-83d5abff5063 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:53.432537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:53.433235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:53.433530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:53.434825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] PATH_INFO: `/v3/users/dc6df949d8114b0e8e3d1ae1a14f12f8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:53.436212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6df949d8114b0e8e3d1ae1a14f12f8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:53.438422 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:53.489277 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:53.502038 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6df949d8114b0e8e3d1ae1a14f12f8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.503430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6df949d8114b0e8e3d1ae1a14f12f8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.504118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6df949d8114b0e8e3d1ae1a14f12f8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.504731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6df949d8114b0e8e3d1ae1a14f12f8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.512160 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5bfb8370-1b3e-4877-877b-be12ac5e27e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6df949d8114b0e8e3d1ae1a14f12f8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.530751 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1373/2751] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:19:53 2022] DELETE /identity/v3/users/dc6df949d8114b0e8e3d1ae1a14f12f8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:53.711590 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:53.713039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:53.730997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e73949-0cb9-438a-9415-03ed07402e3e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:53.794558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:53.795211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:53.795567 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:53.795754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] PATH_INFO: `/v3/projects/5a1bd97db71c4dafb733d670661a632d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:53.796932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1bd97db71c4dafb733d670661a632d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:53.799029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:53.826397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:53.850856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1bd97db71c4dafb733d670661a632d', 'request_id': 'req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.858545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1bd97db71c4dafb733d670661a632d', 'request_id': 'req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.865998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1bd97db71c4dafb733d670661a632d', 'request_id': 'req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.878942 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] The token cache is being invalidate because project 5a1bd97db71c4dafb733d670661a632d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:19:53.879233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.879889 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0056075-ca2b-4ab2-a4cd-a8af6f0c1376 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:53.881422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1379/2752] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:19:53 2022] DELETE /identity/v3/projects/5a1bd97db71c4dafb733d670661a632d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:53.888850 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df098853-a3aa-47ef-8d17-7919c970ba44 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:53.889346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df098853-a3aa-47ef-8d17-7919c970ba44 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:53.889439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df098853-a3aa-47ef-8d17-7919c970ba44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:53.910720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-df098853-a3aa-47ef-8d17-7919c970ba44 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:53.912548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-df098853-a3aa-47ef-8d17-7919c970ba44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:53.978474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1374/2753] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:53.987397 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ddcce-b579-45bd-ba21-0bd78e0842f9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:53.987685 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ddcce-b579-45bd-ba21-0bd78e0842f9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:53.987951 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ddcce-b579-45bd-ba21-0bd78e0842f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:54.007351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1d9ddcce-b579-45bd-ba21-0bd78e0842f9 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:54.008559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1d9ddcce-b579-45bd-ba21-0bd78e0842f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:54.016681 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1380/2754] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:54.021196 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df098853-a3aa-47ef-8d17-7919c970ba44 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:54.032607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df098853-a3aa-47ef-8d17-7919c970ba44 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:54.037354 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670ea90-6f96-4d02-94b6-c2b32fc75f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:54.037997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670ea90-6f96-4d02-94b6-c2b32fc75f47 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:54.038274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670ea90-6f96-4d02-94b6-c2b32fc75f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:54.038554 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670ea90-6f96-4d02-94b6-c2b32fc75f47 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:54.039206 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670ea90-6f96-4d02-94b6-c2b32fc75f47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:19:54.039512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670ea90-6f96-4d02-94b6-c2b32fc75f47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:54.041109 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670ea90-6f96-4d02-94b6-c2b32fc75f47 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:54.053585 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1375/2755] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:19:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:19:54.058276 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ddcce-b579-45bd-ba21-0bd78e0842f9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:54.067716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ddcce-b579-45bd-ba21-0bd78e0842f9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:54.072333 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287cd54-4445-4b9a-a3ea-3e71d1b64690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:54.072930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287cd54-4445-4b9a-a3ea-3e71d1b64690 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:54.073202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287cd54-4445-4b9a-a3ea-3e71d1b64690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:54.073474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287cd54-4445-4b9a-a3ea-3e71d1b64690 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:54.074129 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287cd54-4445-4b9a-a3ea-3e71d1b64690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:54.076554 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287cd54-4445-4b9a-a3ea-3e71d1b64690 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:54.106589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1381/2756] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:54 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:19:54.110968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670ea90-6f96-4d02-94b6-c2b32fc75f47 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:54.120940 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670ea90-6f96-4d02-94b6-c2b32fc75f47 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:54.177789 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ec5ef-6826-483b-a305-e91fcb1a75db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:54.178445 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ec5ef-6826-483b-a305-e91fcb1a75db admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:54.178725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ec5ef-6826-483b-a305-e91fcb1a75db admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:54.179000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ec5ef-6826-483b-a305-e91fcb1a75db admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:54.179638 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ec5ef-6826-483b-a305-e91fcb1a75db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:54.181547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ec5ef-6826-483b-a305-e91fcb1a75db admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:54.204272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1376/2757] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:54 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:19:54.208683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287cd54-4445-4b9a-a3ea-3e71d1b64690 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:54.219749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287cd54-4445-4b9a-a3ea-3e71d1b64690 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:54.224381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af5fe5-c0c1-47ba-8bf8-17acb3e8c59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:54.224969 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af5fe5-c0c1-47ba-8bf8-17acb3e8c59d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:54.225244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af5fe5-c0c1-47ba-8bf8-17acb3e8c59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:54.225513 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1af5fe5-c0c1-47ba-8bf8-17acb3e8c59d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:54.226188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af5fe5-c0c1-47ba-8bf8-17acb3e8c59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:54.227776 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1af5fe5-c0c1-47ba-8bf8-17acb3e8c59d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:54.238710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1382/2758] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:54.243121 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ec5ef-6826-483b-a305-e91fcb1a75db admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:54.254236 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ec5ef-6826-483b-a305-e91fcb1a75db admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:54.258886 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd52e4-743d-40ca-8340-56d0f75ff11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:54.259482 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd52e4-743d-40ca-8340-56d0f75ff11e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:54.259764 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd52e4-743d-40ca-8340-56d0f75ff11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:54.260070 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd52e4-743d-40ca-8340-56d0f75ff11e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:54.260680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd52e4-743d-40ca-8340-56d0f75ff11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:54.262429 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd52e4-743d-40ca-8340-56d0f75ff11e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:54.273627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1377/2759] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:19:54.278912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af5fe5-c0c1-47ba-8bf8-17acb3e8c59d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:54.289040 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1af5fe5-c0c1-47ba-8bf8-17acb3e8c59d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:54.294645 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484616f-0b00-4c22-be66-9949fb49774b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:54.295294 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484616f-0b00-4c22-be66-9949fb49774b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:54.295573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484616f-0b00-4c22-be66-9949fb49774b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:54.295868 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484616f-0b00-4c22-be66-9949fb49774b admin admin] PATH_INFO: `/v3/projects/b46a5c40348e4f7cafe103397140c716/users/35035fe95f754a7e9c74b0ceedd8e7a2/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:54.312899 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484616f-0b00-4c22-be66-9949fb49774b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b46a5c40348e4f7cafe103397140c716, user_id=35035fe95f754a7e9c74b0ceedd8e7a2, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:54.315508 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484616f-0b00-4c22-be66-9949fb49774b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:54.330185 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1383/2760] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:54 2022] PUT /identity/v3/projects/b46a5c40348e4f7cafe103397140c716/users/35035fe95f754a7e9c74b0ceedd8e7a2/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:54.340439 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0130b-3dac-4d33-8102-5db6234dd6a9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:54.340725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0130b-3dac-4d33-8102-5db6234dd6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:54.341025 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0130b-3dac-4d33-8102-5db6234dd6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:54.370373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c5c0130b-3dac-4d33-8102-5db6234dd6a9 None None] MFA Rules not processed for user `35035fe95f754a7e9c74b0ceedd8e7a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:54.372215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c5c0130b-3dac-4d33-8102-5db6234dd6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:54.384448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:54.438976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1378/2761] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:54.450295 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484616f-0b00-4c22-be66-9949fb49774b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:54.451984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2484616f-0b00-4c22-be66-9949fb49774b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:54.509989 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484616f-0b00-4c22-be66-9949fb49774b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:54.514041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06fd5c-a96c-4971-ab59-cd270eb28008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:54.514649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06fd5c-a96c-4971-ab59-cd270eb28008 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:54.514927 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06fd5c-a96c-4971-ab59-cd270eb28008 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:54.515192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06fd5c-a96c-4971-ab59-cd270eb28008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:54.524233 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06fd5c-a96c-4971-ab59-cd270eb28008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:54.525515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d06fd5c-a96c-4971-ab59-cd270eb28008 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:54.535290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1384/2762] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:56.518790 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c5702-adf8-49e0-b772-30d5f1c3b41a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:56.519352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c5702-adf8-49e0-b772-30d5f1c3b41a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:56.519352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c5702-adf8-49e0-b772-30d5f1c3b41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:56.538756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-548c5702-adf8-49e0-b772-30d5f1c3b41a None None] MFA Rules not processed for user `d2749b3c8da94ce1b4af6c708a2f9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:56.539965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-548c5702-adf8-49e0-b772-30d5f1c3b41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:56.594732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1379/2763] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:56.798730 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06fd5c-a96c-4971-ab59-cd270eb28008 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:56.800178 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0d06fd5c-a96c-4971-ab59-cd270eb28008 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:56.857256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:56.872935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d06fd5c-a96c-4971-ab59-cd270eb28008 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:56.877486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:56.878136 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:56.878415 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:56.879199 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] PATH_INFO: `/v3/users/d302c438405142c5af416787d74c41e1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:56.880350 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=d302c438405142c5af416787d74c41e1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:56.882320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:56.932307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd302c438405142c5af416787d74c41e1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:56.933541 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd302c438405142c5af416787d74c41e1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:56.934174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd302c438405142c5af416787d74c41e1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:56.941631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd302c438405142c5af416787d74c41e1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:56.955213 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd302c438405142c5af416787d74c41e1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:56.968560 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1385/2764] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:19:56 2022] DELETE /identity/v3/users/d302c438405142c5af416787d74c41e1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:56.973163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43964eb-df55-40cd-8192-85733a8fcfb0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:56.973444 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43964eb-df55-40cd-8192-85733a8fcfb0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:56.973715 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43964eb-df55-40cd-8192-85733a8fcfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:56.993648 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a43964eb-df55-40cd-8192-85733a8fcfb0 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:56.994932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a43964eb-df55-40cd-8192-85733a8fcfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:57.064153 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1380/2765] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:57.074032 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:57.076331 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:57.122889 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43964eb-df55-40cd-8192-85733a8fcfb0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:57.124409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a43964eb-df55-40cd-8192-85733a8fcfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:57.137908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43964eb-df55-40cd-8192-85733a8fcfb0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:57.143283 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:57.143658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:57.144007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:57.144350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] PATH_INFO: `/v3/users/35035fe95f754a7e9c74b0ceedd8e7a2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:57.145665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] RBAC: Authorizing `identity:delete_user(user_id=35035fe95f754a7e9c74b0ceedd8e7a2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:57.145881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c951e-0f68-4707-ba8c-5a8a4b5716ae admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:57.148247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:57.149817 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a08c80-fa90-439d-8c04-d0fcd75aad79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:57.150427 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a08c80-fa90-439d-8c04-d0fcd75aad79 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:57.150707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a08c80-fa90-439d-8c04-d0fcd75aad79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:57.150973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a08c80-fa90-439d-8c04-d0fcd75aad79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:57.161701 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a08c80-fa90-439d-8c04-d0fcd75aad79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:57.163175 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a08c80-fa90-439d-8c04-d0fcd75aad79 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:57.175228 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1386/2766] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:57.207738 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35035fe95f754a7e9c74b0ceedd8e7a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:57.217480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35035fe95f754a7e9c74b0ceedd8e7a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:57.218602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35035fe95f754a7e9c74b0ceedd8e7a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:57.219448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35035fe95f754a7e9c74b0ceedd8e7a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:57.221755 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:57.230160 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35035fe95f754a7e9c74b0ceedd8e7a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:57.259487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1381/2767] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:19:57 2022] DELETE /identity/v3/users/35035fe95f754a7e9c74b0ceedd8e7a2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:57.264573 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecf2d6-a067-42e9-ac39-ee242a676fc3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:57.264867 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecf2d6-a067-42e9-ac39-ee242a676fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:57.265145 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ecf2d6-a067-42e9-ac39-ee242a676fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:57.286707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b8ecf2d6-a067-42e9-ac39-ee242a676fc3 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:57.287934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8ecf2d6-a067-42e9-ac39-ee242a676fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:57.378067 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1387/2768] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:57.389505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:57.391357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:57.458363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecf2d6-a067-42e9-ac39-ee242a676fc3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:57.460312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8ecf2d6-a067-42e9-ac39-ee242a676fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:57.477692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:57.481520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11a747-7d45-4061-8c53-cd30818b933f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:57.490423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e58301-2948-4342-9107-13b48ff87f1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:57.491890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e58301-2948-4342-9107-13b48ff87f1b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:57.492186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e58301-2948-4342-9107-13b48ff87f1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:57.492462 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e58301-2948-4342-9107-13b48ff87f1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:57.507481 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e58301-2948-4342-9107-13b48ff87f1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:57.509934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e58301-2948-4342-9107-13b48ff87f1b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:57.532936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1382/2769] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:57.566841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ecf2d6-a067-42e9-ac39-ee242a676fc3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:57.573843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1e87ba-7aef-4edf-b1fa-e94488723650 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:57.574912 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e87ba-7aef-4edf-b1fa-e94488723650 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:57.575238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e87ba-7aef-4edf-b1fa-e94488723650 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:57.575672 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1e87ba-7aef-4edf-b1fa-e94488723650 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:57.577916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2e1e87ba-7aef-4edf-b1fa-e94488723650 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:57.593680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e87ba-7aef-4edf-b1fa-e94488723650 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:57.595472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1e87ba-7aef-4edf-b1fa-e94488723650 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:57.630291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1388/2770] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:19:57.644991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e58301-2948-4342-9107-13b48ff87f1b service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:57.646776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c9e58301-2948-4342-9107-13b48ff87f1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:57.665845 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e58301-2948-4342-9107-13b48ff87f1b service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:57.671670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:57.672533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:57.672968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:57.674146 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] PATH_INFO: `/v3/projects/ec778aebbca44642b6e1ccde914af29e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:57.676380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec778aebbca44642b6e1ccde914af29e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:57.679110 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:57.747533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec778aebbca44642b6e1ccde914af29e', 'request_id': 'req-c8717d43-5930-4234-a970-82576cc83e8f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:57.754605 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec778aebbca44642b6e1ccde914af29e', 'request_id': 'req-c8717d43-5930-4234-a970-82576cc83e8f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:57.762792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec778aebbca44642b6e1ccde914af29e', 'request_id': 'req-c8717d43-5930-4234-a970-82576cc83e8f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:57.780160 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] The token cache is being invalidate because project ec778aebbca44642b6e1ccde914af29e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:19:57.780447 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:57.781227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:57.783000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1383/2771] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:19:57 2022] DELETE /identity/v3/projects/ec778aebbca44642b6e1ccde914af29e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:57.788688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b224ec-df51-417d-ac97-3ede11296e36 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:57.788978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b224ec-df51-417d-ac97-3ede11296e36 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:57.789251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b224ec-df51-417d-ac97-3ede11296e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:57.817112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-30b224ec-df51-417d-ac97-3ede11296e36 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:57.818463 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-30b224ec-df51-417d-ac97-3ede11296e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:57.870029 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:57.872332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:57.967041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:57.972332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8717d43-5930-4234-a970-82576cc83e8f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:57.980298 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:57.981129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:57.981641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:57.982411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] PATH_INFO: `/v3/projects/b46a5c40348e4f7cafe103397140c716` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:57.984285 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b46a5c40348e4f7cafe103397140c716)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:57.986487 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.044356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1389/2772] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:58.047086 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b46a5c40348e4f7cafe103397140c716', 'request_id': 'req-affbb48e-115c-4aa6-9830-f7cbfe418a7a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:58.054235 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b46a5c40348e4f7cafe103397140c716', 'request_id': 'req-affbb48e-115c-4aa6-9830-f7cbfe418a7a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:58.054445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e403c-4e40-4a2a-a455-b5f194d5b6fb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.054700 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e403c-4e40-4a2a-a455-b5f194d5b6fb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.055063 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e403c-4e40-4a2a-a455-b5f194d5b6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.061601 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b46a5c40348e4f7cafe103397140c716', 'request_id': 'req-affbb48e-115c-4aa6-9830-f7cbfe418a7a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:58.079407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] The token cache is being invalidate because project b46a5c40348e4f7cafe103397140c716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:19:58.080100 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:58.080666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c82e403c-4e40-4a2a-a455-b5f194d5b6fb None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:58.080842 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affbb48e-115c-4aa6-9830-f7cbfe418a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:19:58.083462 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c82e403c-4e40-4a2a-a455-b5f194d5b6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:58.083941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1384/2773] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:19:57 2022] DELETE /identity/v3/projects/b46a5c40348e4f7cafe103397140c716 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:19:58.088880 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222626e-35ee-40ef-87aa-a7bd04b93ce8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.089357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222626e-35ee-40ef-87aa-a7bd04b93ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.090612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222626e-35ee-40ef-87aa-a7bd04b93ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.113826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6222626e-35ee-40ef-87aa-a7bd04b93ce8 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:58.114776 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6222626e-35ee-40ef-87aa-a7bd04b93ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:58.154493 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1390/2774] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:58.158857 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e403c-4e40-4a2a-a455-b5f194d5b6fb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.171392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e403c-4e40-4a2a-a455-b5f194d5b6fb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.176273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672545b-fc09-41c1-af5e-bc75ae75ac5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.176890 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672545b-fc09-41c1-af5e-bc75ae75ac5d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.177173 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672545b-fc09-41c1-af5e-bc75ae75ac5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.177548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7672545b-fc09-41c1-af5e-bc75ae75ac5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.178327 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672545b-fc09-41c1-af5e-bc75ae75ac5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:19:58.178635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672545b-fc09-41c1-af5e-bc75ae75ac5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.180230 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7672545b-fc09-41c1-af5e-bc75ae75ac5d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.192754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1391/2775] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:19:58.197341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672545b-fc09-41c1-af5e-bc75ae75ac5d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.201915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1385/2776] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:58.207300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42748ec8-6f75-4a23-8ce1-c8b74a6197e3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.207667 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42748ec8-6f75-4a23-8ce1-c8b74a6197e3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.207954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42748ec8-6f75-4a23-8ce1-c8b74a6197e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.208483 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7672545b-fc09-41c1-af5e-bc75ae75ac5d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.214656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07219697-076a-4991-a388-debfe21d4a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.216034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07219697-076a-4991-a388-debfe21d4a9b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.216627 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07219697-076a-4991-a388-debfe21d4a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.216960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07219697-076a-4991-a388-debfe21d4a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.217797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07219697-076a-4991-a388-debfe21d4a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.220680 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07219697-076a-4991-a388-debfe21d4a9b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.229335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-42748ec8-6f75-4a23-8ce1-c8b74a6197e3 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:58.231250 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-42748ec8-6f75-4a23-8ce1-c8b74a6197e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:58.241346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1386/2777] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:58.247339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42748ec8-6f75-4a23-8ce1-c8b74a6197e3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.255445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1392/2778] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:19:58.259479 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07219697-076a-4991-a388-debfe21d4a9b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.259890 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42748ec8-6f75-4a23-8ce1-c8b74a6197e3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.272414 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:58.331811 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff789559-9c76-45d9-8688-09227dc6a9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.332843 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff789559-9c76-45d9-8688-09227dc6a9bf admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.333314 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff789559-9c76-45d9-8688-09227dc6a9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.333882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff789559-9c76-45d9-8688-09227dc6a9bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.334711 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff789559-9c76-45d9-8688-09227dc6a9bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:19:58.335020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff789559-9c76-45d9-8688-09227dc6a9bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.336639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff789559-9c76-45d9-8688-09227dc6a9bf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.336796 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07219697-076a-4991-a388-debfe21d4a9b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.342609 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81145345-363b-4edf-a6cb-4cfe00116f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.343228 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81145345-363b-4edf-a6cb-4cfe00116f92 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.343510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81145345-363b-4edf-a6cb-4cfe00116f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.343788 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81145345-363b-4edf-a6cb-4cfe00116f92 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.344548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81145345-363b-4edf-a6cb-4cfe00116f92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.346500 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81145345-363b-4edf-a6cb-4cfe00116f92 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.350137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1387/2779] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:19:58.354401 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff789559-9c76-45d9-8688-09227dc6a9bf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.366046 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff789559-9c76-45d9-8688-09227dc6a9bf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.369464 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1393/2780] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/users => generated 290 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:19:58.371894 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6078e1-f565-4d7d-b2eb-430c5679366f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.372526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6078e1-f565-4d7d-b2eb-430c5679366f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.372806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6078e1-f565-4d7d-b2eb-430c5679366f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.373215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6078e1-f565-4d7d-b2eb-430c5679366f admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.374203 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6078e1-f565-4d7d-b2eb-430c5679366f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.374594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81145345-363b-4edf-a6cb-4cfe00116f92 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.376134 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6078e1-f565-4d7d-b2eb-430c5679366f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.385273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81145345-363b-4edf-a6cb-4cfe00116f92 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.390834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa60da2-d32f-4b38-8210-d49d9703eff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.391518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa60da2-d32f-4b38-8210-d49d9703eff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.392196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa60da2-d32f-4b38-8210-d49d9703eff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.392474 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa60da2-d32f-4b38-8210-d49d9703eff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.392959 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:58.393243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa60da2-d32f-4b38-8210-d49d9703eff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.395060 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa60da2-d32f-4b38-8210-d49d9703eff5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.406599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1394/2781] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:58.410737 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa60da2-d32f-4b38-8210-d49d9703eff5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.413919 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1388/2782] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:19:58.414785 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6078e1-f565-4d7d-b2eb-430c5679366f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.420774 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa60da2-d32f-4b38-8210-d49d9703eff5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.479444 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263865a-5111-4683-8589-d941632c9314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.480050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263865a-5111-4683-8589-d941632c9314 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.480324 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263865a-5111-4683-8589-d941632c9314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.480648 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263865a-5111-4683-8589-d941632c9314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.481291 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263865a-5111-4683-8589-d941632c9314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.481662 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6078e1-f565-4d7d-b2eb-430c5679366f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.482911 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263865a-5111-4683-8589-d941632c9314 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.486780 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f5432-298f-4e4f-92d9-3618193ce997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.487441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f5432-298f-4e4f-92d9-3618193ce997 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.487731 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f5432-298f-4e4f-92d9-3618193ce997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.488010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f5432-298f-4e4f-92d9-3618193ce997 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.488639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f5432-298f-4e4f-92d9-3618193ce997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.490616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f5432-298f-4e4f-92d9-3618193ce997 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.493194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1395/2783] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:58.498052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263865a-5111-4683-8589-d941632c9314 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.508870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263865a-5111-4683-8589-d941632c9314 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.513230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1389/2784] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:19:58.514545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1ed08-f5b3-4cec-8d05-95c638bb3d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.515188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1ed08-f5b3-4cec-8d05-95c638bb3d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.515471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1ed08-f5b3-4cec-8d05-95c638bb3d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.515745 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1ed08-f5b3-4cec-8d05-95c638bb3d82 admin admin] PATH_INFO: `/v3/projects/d5be11d5ab14425e99324eccb69ad592/users/3efb16412f9545e3b0d56482127e52ec/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.517415 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f5432-298f-4e4f-92d9-3618193ce997 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.528122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f5432-298f-4e4f-92d9-3618193ce997 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.531893 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1ed08-f5b3-4cec-8d05-95c638bb3d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5be11d5ab14425e99324eccb69ad592, user_id=3efb16412f9545e3b0d56482127e52ec, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.533037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d994f4-d279-4b27-abf4-f968e5b75e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.533645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d994f4-d279-4b27-abf4-f968e5b75e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.533963 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d994f4-d279-4b27-abf4-f968e5b75e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.534227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d994f4-d279-4b27-abf4-f968e5b75e4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.534874 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d994f4-d279-4b27-abf4-f968e5b75e4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.535469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd1ed08-f5b3-4cec-8d05-95c638bb3d82 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.536579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d994f4-d279-4b27-abf4-f968e5b75e4e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.551645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1390/2785] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:58.555749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d994f4-d279-4b27-abf4-f968e5b75e4e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.556332 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1396/2786] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:58 2022] PUT /identity/v3/projects/d5be11d5ab14425e99324eccb69ad592/users/3efb16412f9545e3b0d56482127e52ec/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:19:58.568255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d994f4-d279-4b27-abf4-f968e5b75e4e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.570418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627f806-595f-4bbe-a877-01740c0e4d2a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.571381 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627f806-595f-4bbe-a877-01740c0e4d2a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.571586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627f806-595f-4bbe-a877-01740c0e4d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.609938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b627f806-595f-4bbe-a877-01740c0e4d2a None None] MFA Rules not processed for user `3efb16412f9545e3b0d56482127e52ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:58.611176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b627f806-595f-4bbe-a877-01740c0e4d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:58.643682 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7f460-b7b8-4f39-a4fa-732e26eb9cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.644560 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7f460-b7b8-4f39-a4fa-732e26eb9cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.645431 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7f460-b7b8-4f39-a4fa-732e26eb9cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.645898 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7f460-b7b8-4f39-a4fa-732e26eb9cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.647003 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7f460-b7b8-4f39-a4fa-732e26eb9cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.649119 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7f460-b7b8-4f39-a4fa-732e26eb9cd3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.665208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1391/2787] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:58 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:58.671397 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7f460-b7b8-4f39-a4fa-732e26eb9cd3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.684192 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7f460-b7b8-4f39-a4fa-732e26eb9cd3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.685906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:58.692630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0ae84-43e6-4031-ad29-664eadcf365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.693258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0ae84-43e6-4031-ad29-664eadcf365a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.695191 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0ae84-43e6-4031-ad29-664eadcf365a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.697163 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0ae84-43e6-4031-ad29-664eadcf365a admin admin] PATH_INFO: `/v3/projects/d6295d3980694efd879eff5e64d848b2/users/dc6e32aa37f445d1a74df1b2dd98aa2f/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.716799 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0ae84-43e6-4031-ad29-664eadcf365a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6295d3980694efd879eff5e64d848b2, user_id=dc6e32aa37f445d1a74df1b2dd98aa2f, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.723497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0ae84-43e6-4031-ad29-664eadcf365a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.731501 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1397/2788] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:58.737128 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627f806-595f-4bbe-a877-01740c0e4d2a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.747861 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1392/2789] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:58 2022] PUT /identity/v3/projects/d6295d3980694efd879eff5e64d848b2/users/dc6e32aa37f445d1a74df1b2dd98aa2f/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:19:58.758498 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c03d7-229a-41f7-81f3-595868346fd3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.759333 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c03d7-229a-41f7-81f3-595868346fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.759830 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c03d7-229a-41f7-81f3-595868346fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.761320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627f806-595f-4bbe-a877-01740c0e4d2a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.821613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b95c03d7-229a-41f7-81f3-595868346fd3 None None] MFA Rules not processed for user `dc6e32aa37f445d1a74df1b2dd98aa2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:58.823238 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b95c03d7-229a-41f7-81f3-595868346fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:58.843574 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34ef26-162d-43d8-a6a6-a35f5e612dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.844433 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34ef26-162d-43d8-a6a6-a35f5e612dcc admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.844810 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34ef26-162d-43d8-a6a6-a35f5e612dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.845823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34ef26-162d-43d8-a6a6-a35f5e612dcc admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.845823 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34ef26-162d-43d8-a6a6-a35f5e612dcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.849997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34ef26-162d-43d8-a6a6-a35f5e612dcc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:58.866757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:58.889888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1398/2790] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/projects => generated 381 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:19:58.891610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34ef26-162d-43d8-a6a6-a35f5e612dcc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.916969 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1393/2791] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:19:58.928862 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c03d7-229a-41f7-81f3-595868346fd3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:58.930390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b95c03d7-229a-41f7-81f3-595868346fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:58.976031 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34ef26-162d-43d8-a6a6-a35f5e612dcc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:58.980994 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721bf10-f8fa-4f54-8635-6a89df17af7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:58.981603 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721bf10-f8fa-4f54-8635-6a89df17af7d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:58.981922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721bf10-f8fa-4f54-8635-6a89df17af7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:58.982194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721bf10-f8fa-4f54-8635-6a89df17af7d admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:58.982820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721bf10-f8fa-4f54-8635-6a89df17af7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:58.984744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9721bf10-f8fa-4f54-8635-6a89df17af7d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:59.007606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1399/2792] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:19:58 2022] POST /identity/v3/users => generated 297 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:19:59.011597 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721bf10-f8fa-4f54-8635-6a89df17af7d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:59.022886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9721bf10-f8fa-4f54-8635-6a89df17af7d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:59.027101 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c03d7-229a-41f7-81f3-595868346fd3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:59.029072 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2621e73-1d75-4a6b-9401-5f04fb208e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:59.029687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2621e73-1d75-4a6b-9401-5f04fb208e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:59.030006 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2621e73-1d75-4a6b-9401-5f04fb208e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:59.030281 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2621e73-1d75-4a6b-9401-5f04fb208e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:59.030930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2621e73-1d75-4a6b-9401-5f04fb208e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:59.032542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2621e73-1d75-4a6b-9401-5f04fb208e4c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:59.042792 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1400/2793] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:59.047582 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2621e73-1d75-4a6b-9401-5f04fb208e4c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:59.057215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2621e73-1d75-4a6b-9401-5f04fb208e4c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:59.062521 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f67d4-0db4-4efc-acb2-146794dbcb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:59.063956 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f67d4-0db4-4efc-acb2-146794dbcb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:59.064237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f67d4-0db4-4efc-acb2-146794dbcb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:59.064510 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f67d4-0db4-4efc-acb2-146794dbcb40 admin admin] PATH_INFO: `/v3/projects/8475d3811c6641018d0db0aecc6df391/users/f8577b947fbd4874b5921ac60a53fcba/roles/8b350845f331423cb0494e3449b58310` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:59.079275 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f4b6e-68fa-4f02-ade4-4b9268abfe70 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b957b367ddf2483f84056ccec57fc515', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:59.080015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f4b6e-68fa-4f02-ade4-4b9268abfe70 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:59.080293 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f4b6e-68fa-4f02-ade4-4b9268abfe70 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:59.080416 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f67d4-0db4-4efc-acb2-146794dbcb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8475d3811c6641018d0db0aecc6df391, user_id=f8577b947fbd4874b5921ac60a53fcba, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:59.080690 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4f4b6e-68fa-4f02-ade4-4b9268abfe70 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:59.083079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f67d4-0db4-4efc-acb2-146794dbcb40 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:59.097874 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1401/2794] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:59 2022] PUT /identity/v3/projects/8475d3811c6641018d0db0aecc6df391/users/f8577b947fbd4874b5921ac60a53fcba/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:19:59.102084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f67d4-0db4-4efc-acb2-146794dbcb40 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:59.134078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:59.146216 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f4b6e-68fa-4f02-ade4-4b9268abfe70 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:59.147734 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4f4b6e-68fa-4f02-ade4-4b9268abfe70 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:59.157476 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1394/2795] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:19:59.170546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f67d4-0db4-4efc-acb2-146794dbcb40 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:59.175119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972f16b-eac6-422f-be93-aee47bab8e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:59.175733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972f16b-eac6-422f-be93-aee47bab8e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:59.176238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972f16b-eac6-422f-be93-aee47bab8e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:59.176238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972f16b-eac6-422f-be93-aee47bab8e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:59.176895 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972f16b-eac6-422f-be93-aee47bab8e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:59.178518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972f16b-eac6-422f-be93-aee47bab8e64 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:59.189540 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1402/2796] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:19:59.194563 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f4b6e-68fa-4f02-ade4-4b9268abfe70 service glance] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:59.206101 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4f4b6e-68fa-4f02-ade4-4b9268abfe70 service glance] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:59.211493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9f866-e0b1-422e-a66a-eb70e1a43876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:59.212276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9f866-e0b1-422e-a66a-eb70e1a43876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:59.212558 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9f866-e0b1-422e-a66a-eb70e1a43876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:59.212964 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9f866-e0b1-422e-a66a-eb70e1a43876 admin admin] PATH_INFO: `/v3/projects/8475d3811c6641018d0db0aecc6df391/users/f8577b947fbd4874b5921ac60a53fcba/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:59.215686 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9f866-e0b1-422e-a66a-eb70e1a43876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8475d3811c6641018d0db0aecc6df391, user_id=f8577b947fbd4874b5921ac60a53fcba, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:59.218714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9f866-e0b1-422e-a66a-eb70e1a43876 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:59.229680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1395/2797] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:19:59 2022] PUT /identity/v3/projects/8475d3811c6641018d0db0aecc6df391/users/f8577b947fbd4874b5921ac60a53fcba/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:19:59.241932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a39ff-d5f9-4fd8-bda5-e7334404663a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:59.242549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a39ff-d5f9-4fd8-bda5-e7334404663a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:59.242697 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a39ff-d5f9-4fd8-bda5-e7334404663a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:59.274019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d15a39ff-d5f9-4fd8-bda5-e7334404663a None None] MFA Rules not processed for user `f8577b947fbd4874b5921ac60a53fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:19:59.275318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d15a39ff-d5f9-4fd8-bda5-e7334404663a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:59.349709 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1403/2798] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:19:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:19:59.362506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9f866-e0b1-422e-a66a-eb70e1a43876 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:59.364230 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1de9f866-e0b1-422e-a66a-eb70e1a43876 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:19:59.379043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:19:59.426661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9f866-e0b1-422e-a66a-eb70e1a43876 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:59.430270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8f742-247c-4de4-a993-0d6d5eb2060b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:59.430882 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8f742-247c-4de4-a993-0d6d5eb2060b service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:59.431160 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8f742-247c-4de4-a993-0d6d5eb2060b service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:59.431434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa8f742-247c-4de4-a993-0d6d5eb2060b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:59.442971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8f742-247c-4de4-a993-0d6d5eb2060b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:59.444863 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa8f742-247c-4de4-a993-0d6d5eb2060b service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:59.455768 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1396/2799] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:19:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:19:59.816816 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a39ff-d5f9-4fd8-bda5-e7334404663a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:59.835451 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a39ff-d5f9-4fd8-bda5-e7334404663a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:59.843320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9169d-6d41-4db8-b690-cf39b52e8ca0 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8577b947fbd4874b5921ac60a53fcba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8475d3811c6641018d0db0aecc6df391', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:59.844200 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9169d-6d41-4db8-b690-cf39b52e8ca0 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:59.845119 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9169d-6d41-4db8-b690-cf39b52e8ca0 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:59.845712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9169d-6d41-4db8-b690-cf39b52e8ca0 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:59.846915 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9169d-6d41-4db8-b690-cf39b52e8ca0 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:19:59.847243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9169d-6d41-4db8-b690-cf39b52e8ca0 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:59.849997 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac9169d-6d41-4db8-b690-cf39b52e8ca0 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:19:59.869831 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1404/2800] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:19:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:19:59.872489 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8f742-247c-4de4-a993-0d6d5eb2060b service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:19:59.894915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa8f742-247c-4de4-a993-0d6d5eb2060b service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:19:59.907128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d2898-1138-405d-8d8c-4c405a215954 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8577b947fbd4874b5921ac60a53fcba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8475d3811c6641018d0db0aecc6df391', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:19:59.907782 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d2898-1138-405d-8d8c-4c405a215954 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:19:59.908065 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d2898-1138-405d-8d8c-4c405a215954 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:19:59.908348 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d2898-1138-405d-8d8c-4c405a215954 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:19:59.909000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d2898-1138-405d-8d8c-4c405a215954 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:19:59.912124 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d2898-1138-405d-8d8c-4c405a215954 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:00.008441 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1397/2801] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:19:59 2022] POST /identity/v3/projects => generated 354 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:20:00.010127 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9169d-6d41-4db8-b690-cf39b52e8ca0 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:00.024555 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac9169d-6d41-4db8-b690-cf39b52e8ca0 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:00.104195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224cb55-0f40-4fb3-99b8-fa86a0c4ee1f tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8577b947fbd4874b5921ac60a53fcba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8475d3811c6641018d0db0aecc6df391', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:00.105716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224cb55-0f40-4fb3-99b8-fa86a0c4ee1f tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:00.106615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224cb55-0f40-4fb3-99b8-fa86a0c4ee1f tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:00.107187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224cb55-0f40-4fb3-99b8-fa86a0c4ee1f tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:00.108196 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224cb55-0f40-4fb3-99b8-fa86a0c4ee1f tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:20:00.109431 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224cb55-0f40-4fb3-99b8-fa86a0c4ee1f tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:00.120854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224cb55-0f40-4fb3-99b8-fa86a0c4ee1f tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:00.136280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1405/2802] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:20:00.178368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d2898-1138-405d-8d8c-4c405a215954 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:00.193159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d2898-1138-405d-8d8c-4c405a215954 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:00.199812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f8577b947fbd4874b5921ac60a53fcba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8475d3811c6641018d0db0aecc6df391', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:00.200781 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:00.201302 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:00.201728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] PATH_INFO: `/v3/projects/3b7d39e963c84daf8cb166f63b7392f1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:00.203375 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3b7d39e963c84daf8cb166f63b7392f1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:00.206579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:00.275434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b7d39e963c84daf8cb166f63b7392f1', 'request_id': 'req-bab2fe56-c1c2-48fa-8240-a81cad340055'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:00.295708 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b7d39e963c84daf8cb166f63b7392f1', 'request_id': 'req-bab2fe56-c1c2-48fa-8240-a81cad340055'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:00.310669 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b7d39e963c84daf8cb166f63b7392f1', 'request_id': 'req-bab2fe56-c1c2-48fa-8240-a81cad340055'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:00.328533 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] The token cache is being invalidate because project 3b7d39e963c84daf8cb166f63b7392f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:20:00.329166 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:00.331060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bab2fe56-c1c2-48fa-8240-a81cad340055 tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:00.343000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1398/2803] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:20:00 2022] DELETE /identity/v3/projects/3b7d39e963c84daf8cb166f63b7392f1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:00.623576 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224cb55-0f40-4fb3-99b8-fa86a0c4ee1f tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:00.627256 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9224cb55-0f40-4fb3-99b8-fa86a0c4ee1f tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:00.689139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:00.737353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224cb55-0f40-4fb3-99b8-fa86a0c4ee1f tempest-VolumeQuotaClassesTest-1805356279 tempest-VolumeQuotaClassesTest-1805356279-project-admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:00.800193 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:00.800872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:00.801146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:00.801552 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] PATH_INFO: `/v3/users/3efb16412f9545e3b0d56482127e52ec` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:00.804269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3efb16412f9545e3b0d56482127e52ec)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:00.806382 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:00.869259 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3efb16412f9545e3b0d56482127e52ec'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:00.869953 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3efb16412f9545e3b0d56482127e52ec'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:00.870584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3efb16412f9545e3b0d56482127e52ec'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:00.878422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3efb16412f9545e3b0d56482127e52ec'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:00.891952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3efb16412f9545e3b0d56482127e52ec'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:00.906981 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1406/2804] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:20:00 2022] DELETE /identity/v3/users/3efb16412f9545e3b0d56482127e52ec => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:00.911488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4e0e4-67c8-458b-93ba-6fd7d9dab36b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:00.911974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4e0e4-67c8-458b-93ba-6fd7d9dab36b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:00.912215 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4e0e4-67c8-458b-93ba-6fd7d9dab36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:00.932968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-acc4e0e4-67c8-458b-93ba-6fd7d9dab36b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:00.935162 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-acc4e0e4-67c8-458b-93ba-6fd7d9dab36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:01.009159 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1399/2805] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:01.020367 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:01.022215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:01.108968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4b074-4e2c-4c3d-b7ff-fd127df48b32 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:01.112932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac9241-71fd-41f1-b74b-9fff9eda39e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:01.113539 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac9241-71fd-41f1-b74b-9fff9eda39e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:01.113841 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac9241-71fd-41f1-b74b-9fff9eda39e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:01.114114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ac9241-71fd-41f1-b74b-9fff9eda39e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:01.124420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac9241-71fd-41f1-b74b-9fff9eda39e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:01.125885 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ac9241-71fd-41f1-b74b-9fff9eda39e6 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:01.133188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:01.145194 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1407/2806] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:01.736355 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4e0e4-67c8-458b-93ba-6fd7d9dab36b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:01.738157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-acc4e0e4-67c8-458b-93ba-6fd7d9dab36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:01.751227 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4e0e4-67c8-458b-93ba-6fd7d9dab36b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:01.756469 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:01.757343 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:01.757906 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:01.758290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] PATH_INFO: `/v3/projects/d5be11d5ab14425e99324eccb69ad592` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:01.759764 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5be11d5ab14425e99324eccb69ad592)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:01.762618 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:01.790630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:01.850535 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5be11d5ab14425e99324eccb69ad592', 'request_id': 'req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:01.858822 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5be11d5ab14425e99324eccb69ad592', 'request_id': 'req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:01.865217 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5be11d5ab14425e99324eccb69ad592', 'request_id': 'req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:01.882459 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] The token cache is being invalidate because project d5be11d5ab14425e99324eccb69ad592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:20:01.882760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:01.883521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:01.885757 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1400/2807] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:20:01 2022] DELETE /identity/v3/projects/d5be11d5ab14425e99324eccb69ad592 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:01.890398 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac9241-71fd-41f1-b74b-9fff9eda39e6 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:01.901208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ac9241-71fd-41f1-b74b-9fff9eda39e6 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:01.976307 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:01.976971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:01.977253 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:01.977801 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] PATH_INFO: `/v3/users/f8577b947fbd4874b5921ac60a53fcba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:01.978692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8577b947fbd4874b5921ac60a53fcba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:01.981202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:02.055615 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8577b947fbd4874b5921ac60a53fcba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:02.056183 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8577b947fbd4874b5921ac60a53fcba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:02.056825 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8577b947fbd4874b5921ac60a53fcba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:02.066151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8577b947fbd4874b5921ac60a53fcba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:02.092059 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ff96a95e-ad42-42e3-a3ef-c14ee70695c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8577b947fbd4874b5921ac60a53fcba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:02.104932 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1408/2808] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:20:01 2022] DELETE /identity/v3/users/f8577b947fbd4874b5921ac60a53fcba => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:02.395714 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:02.397528 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:02.413028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8a4e8-dd00-45d0-88f1-3e9cc6f342df admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:02.474733 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:02.475652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:02.475917 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:02.476186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] PATH_INFO: `/v3/projects/8475d3811c6641018d0db0aecc6df391` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:02.477332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=8475d3811c6641018d0db0aecc6df391)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:02.479274 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:02.527873 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8475d3811c6641018d0db0aecc6df391', 'request_id': 'req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:02.534155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8475d3811c6641018d0db0aecc6df391', 'request_id': 'req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:02.539688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8475d3811c6641018d0db0aecc6df391', 'request_id': 'req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:02.552586 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] The token cache is being invalidate because project 8475d3811c6641018d0db0aecc6df391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:20:02.552876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:02.553499 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f310d9f9-f7ae-4b5c-b4d7-d70ccde804ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:02.555016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1401/2809] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:20:02 2022] DELETE /identity/v3/projects/8475d3811c6641018d0db0aecc6df391 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:02.561627 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da25bb6-eb5e-4d81-bfc2-cba02bea639b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:02.561959 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da25bb6-eb5e-4d81-bfc2-cba02bea639b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:02.562238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da25bb6-eb5e-4d81-bfc2-cba02bea639b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:02.583054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7da25bb6-eb5e-4d81-bfc2-cba02bea639b None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:02.584206 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7da25bb6-eb5e-4d81-bfc2-cba02bea639b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:02.648267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1409/2810] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:02.657602 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027f36a-628d-4353-b6ee-9611fe5395fc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:02.658020 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027f36a-628d-4353-b6ee-9611fe5395fc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:02.658296 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027f36a-628d-4353-b6ee-9611fe5395fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:02.679273 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c027f36a-628d-4353-b6ee-9611fe5395fc None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:02.680579 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c027f36a-628d-4353-b6ee-9611fe5395fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:02.692090 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1402/2811] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:02.696238 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da25bb6-eb5e-4d81-bfc2-cba02bea639b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:02.707061 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da25bb6-eb5e-4d81-bfc2-cba02bea639b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:02.712251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93446da-953b-4fab-aece-28e4ab44d3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:02.712870 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93446da-953b-4fab-aece-28e4ab44d3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:02.713146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93446da-953b-4fab-aece-28e4ab44d3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:02.713532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93446da-953b-4fab-aece-28e4ab44d3f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:02.714217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93446da-953b-4fab-aece-28e4ab44d3f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:20:02.714523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93446da-953b-4fab-aece-28e4ab44d3f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:02.716169 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93446da-953b-4fab-aece-28e4ab44d3f3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:02.736279 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1410/2812] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:20:02.741066 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027f36a-628d-4353-b6ee-9611fe5395fc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:02.755616 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027f36a-628d-4353-b6ee-9611fe5395fc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:02.764063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d72daa-011a-4209-890c-b90357113b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:02.764685 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d72daa-011a-4209-890c-b90357113b9f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:02.764965 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d72daa-011a-4209-890c-b90357113b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:02.765243 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d72daa-011a-4209-890c-b90357113b9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:02.765982 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d72daa-011a-4209-890c-b90357113b9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:02.768609 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d72daa-011a-4209-890c-b90357113b9f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:02.804547 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:02.817794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1403/2813] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:20:02 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:20:02.821832 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93446da-953b-4fab-aece-28e4ab44d3f3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:02.837886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93446da-953b-4fab-aece-28e4ab44d3f3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:02.842688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:02.946902 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072c4a7-35ae-418e-b540-0e78e4ad604e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:02.947535 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072c4a7-35ae-418e-b540-0e78e4ad604e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:02.947794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072c4a7-35ae-418e-b540-0e78e4ad604e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:02.948066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4072c4a7-35ae-418e-b540-0e78e4ad604e admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:02.948705 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072c4a7-35ae-418e-b540-0e78e4ad604e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:02.951446 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4072c4a7-35ae-418e-b540-0e78e4ad604e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:02.977741 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1411/2814] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:20:02 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:20:02.982805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d72daa-011a-4209-890c-b90357113b9f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:02.993173 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d72daa-011a-4209-890c-b90357113b9f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:02.998366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671a168-344f-43f8-bef4-35cefe6a5d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:02.999095 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671a168-344f-43f8-bef4-35cefe6a5d22 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:02.999338 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671a168-344f-43f8-bef4-35cefe6a5d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:02.999630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d671a168-344f-43f8-bef4-35cefe6a5d22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.000493 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671a168-344f-43f8-bef4-35cefe6a5d22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.002076 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d671a168-344f-43f8-bef4-35cefe6a5d22 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.013260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1404/2815] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:20:03.017586 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072c4a7-35ae-418e-b540-0e78e4ad604e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:03.030318 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4072c4a7-35ae-418e-b540-0e78e4ad604e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:03.034954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f8452-b40b-4526-a989-488873c52b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:03.035559 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f8452-b40b-4526-a989-488873c52b8f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.035834 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f8452-b40b-4526-a989-488873c52b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.036102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f8452-b40b-4526-a989-488873c52b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.036735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f8452-b40b-4526-a989-488873c52b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.038348 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f8452-b40b-4526-a989-488873c52b8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.050363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1412/2816] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:20:03.054706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671a168-344f-43f8-bef4-35cefe6a5d22 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:03.066336 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d671a168-344f-43f8-bef4-35cefe6a5d22 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:03.071015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a4384-84fd-406b-bc2b-c29debafb313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:03.071681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4384-84fd-406b-bc2b-c29debafb313 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.071956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4384-84fd-406b-bc2b-c29debafb313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.072226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4384-84fd-406b-bc2b-c29debafb313 admin admin] PATH_INFO: `/v3/projects/f9dd808393454ae682a4e6f1425587f0/users/8756b5633e8140b4977245cffa995ccb/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.092118 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a4384-84fd-406b-bc2b-c29debafb313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9dd808393454ae682a4e6f1425587f0, user_id=8756b5633e8140b4977245cffa995ccb, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.094915 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31a4384-84fd-406b-bc2b-c29debafb313 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.107846 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1405/2817] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:20:03 2022] PUT /identity/v3/projects/f9dd808393454ae682a4e6f1425587f0/users/8756b5633e8140b4977245cffa995ccb/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:03.119354 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d09ad-87ab-4232-9f41-3a0d79917d6f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.120217 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d09ad-87ab-4232-9f41-3a0d79917d6f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.120779 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d09ad-87ab-4232-9f41-3a0d79917d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.153078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-930d09ad-87ab-4232-9f41-3a0d79917d6f None None] MFA Rules not processed for user `8756b5633e8140b4977245cffa995ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:03.154604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-930d09ad-87ab-4232-9f41-3a0d79917d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:03.232604 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1413/2818] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:03.240767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a4384-84fd-406b-bc2b-c29debafb313 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:03.318702 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a4384-84fd-406b-bc2b-c29debafb313 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:03.323483 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a029e8a-156f-4005-9b94-a6e621e8b8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:03.324123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a029e8a-156f-4005-9b94-a6e621e8b8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.324407 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a029e8a-156f-4005-9b94-a6e621e8b8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.324680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a029e8a-156f-4005-9b94-a6e621e8b8f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.325467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a029e8a-156f-4005-9b94-a6e621e8b8f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.327503 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a029e8a-156f-4005-9b94-a6e621e8b8f9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.361271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d09ad-87ab-4232-9f41-3a0d79917d6f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:03.362149 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1406/2819] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:20:03 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:20:03.363153 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-930d09ad-87ab-4232-9f41-3a0d79917d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:03.365895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a029e8a-156f-4005-9b94-a6e621e8b8f9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:03.449123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a029e8a-156f-4005-9b94-a6e621e8b8f9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:03.454445 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d09ad-87ab-4232-9f41-3a0d79917d6f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:03.459067 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c0099-c1e6-41b6-8cc8-78b3136f38c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:03.460844 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c0099-c1e6-41b6-8cc8-78b3136f38c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.461225 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c0099-c1e6-41b6-8cc8-78b3136f38c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.461823 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c0099-c1e6-41b6-8cc8-78b3136f38c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.462486 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c0099-c1e6-41b6-8cc8-78b3136f38c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.465497 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:03.466943 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c0099-c1e6-41b6-8cc8-78b3136f38c6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.492973 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1407/2820] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:20:03 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 19 21:20:03.494030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c0099-c1e6-41b6-8cc8-78b3136f38c6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:03.504653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c0099-c1e6-41b6-8cc8-78b3136f38c6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:03.509513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e383-7289-4fb2-aeb6-20af995fb2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:03.510249 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e383-7289-4fb2-aeb6-20af995fb2f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.510537 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e383-7289-4fb2-aeb6-20af995fb2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.510817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e383-7289-4fb2-aeb6-20af995fb2f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.511594 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661e383-7289-4fb2-aeb6-20af995fb2f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.514425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661e383-7289-4fb2-aeb6-20af995fb2f0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.525736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333d9c2-2c8b-40bd-afdd-3269576635b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:03.526387 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333d9c2-2c8b-40bd-afdd-3269576635b4 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.526539 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1408/2821] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:20:03.526665 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333d9c2-2c8b-40bd-afdd-3269576635b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.526939 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333d9c2-2c8b-40bd-afdd-3269576635b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.529048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2333d9c2-2c8b-40bd-afdd-3269576635b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:03.530596 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e383-7289-4fb2-aeb6-20af995fb2f0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:03.540463 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e383-7289-4fb2-aeb6-20af995fb2f0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:03.541778 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333d9c2-2c8b-40bd-afdd-3269576635b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.543364 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333d9c2-2c8b-40bd-afdd-3269576635b4 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.545454 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878be0a-87df-4d02-8a20-07d914ad36d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:03.546128 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878be0a-87df-4d02-8a20-07d914ad36d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.546406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878be0a-87df-4d02-8a20-07d914ad36d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.546680 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878be0a-87df-4d02-8a20-07d914ad36d4 admin admin] PATH_INFO: `/v3/projects/a187d7870f4c4d84ae803a18729f428c/users/4eb92114e0a04f42b2872c0c3399ec5b/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.555280 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1414/2822] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:03.562521 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878be0a-87df-4d02-8a20-07d914ad36d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a187d7870f4c4d84ae803a18729f428c, user_id=4eb92114e0a04f42b2872c0c3399ec5b, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.565244 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878be0a-87df-4d02-8a20-07d914ad36d4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.582677 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1409/2823] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:20:03 2022] PUT /identity/v3/projects/a187d7870f4c4d84ae803a18729f428c/users/4eb92114e0a04f42b2872c0c3399ec5b/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:20:03.587123 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333d9c2-2c8b-40bd-afdd-3269576635b4 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:03.598511 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333d9c2-2c8b-40bd-afdd-3269576635b4 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:03.666735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dd949-f6d2-4a44-a19c-deff36380260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:03.667379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dd949-f6d2-4a44-a19c-deff36380260 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.667656 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dd949-f6d2-4a44-a19c-deff36380260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.668612 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dd949-f6d2-4a44-a19c-deff36380260 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.669197 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dd949-f6d2-4a44-a19c-deff36380260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.670816 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dd949-f6d2-4a44-a19c-deff36380260 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.680447 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1415/2824] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:20:03.685260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878be0a-87df-4d02-8a20-07d914ad36d4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:03.696909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:03.702756 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878be0a-87df-4d02-8a20-07d914ad36d4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:03.710505 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92500fb-2174-4fbb-86d3-8ea69ab09e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:03.711281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92500fb-2174-4fbb-86d3-8ea69ab09e7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.711633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92500fb-2174-4fbb-86d3-8ea69ab09e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.711908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92500fb-2174-4fbb-86d3-8ea69ab09e7f admin admin] PATH_INFO: `/v3/projects/a187d7870f4c4d84ae803a18729f428c/users/4eb92114e0a04f42b2872c0c3399ec5b/roles/8b350845f331423cb0494e3449b58310` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.713928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92500fb-2174-4fbb-86d3-8ea69ab09e7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a187d7870f4c4d84ae803a18729f428c, user_id=4eb92114e0a04f42b2872c0c3399ec5b, role_id=8b350845f331423cb0494e3449b58310)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.717115 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92500fb-2174-4fbb-86d3-8ea69ab09e7f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.728015 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1410/2825] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:20:03 2022] PUT /identity/v3/projects/a187d7870f4c4d84ae803a18729f428c/users/4eb92114e0a04f42b2872c0c3399ec5b/roles/8b350845f331423cb0494e3449b58310 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 19 21:20:03.740016 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2f558-b1b9-4910-9ff2-027966079802 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.740401 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2f558-b1b9-4910-9ff2-027966079802 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.740661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2f558-b1b9-4910-9ff2-027966079802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.771084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-75a2f558-b1b9-4910-9ff2-027966079802 None None] MFA Rules not processed for user `4eb92114e0a04f42b2872c0c3399ec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:03.772313 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-75a2f558-b1b9-4910-9ff2-027966079802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:03.847578 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1416/2826] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:03.859479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92500fb-2174-4fbb-86d3-8ea69ab09e7f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:03.861805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c92500fb-2174-4fbb-86d3-8ea69ab09e7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:03.932308 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92500fb-2174-4fbb-86d3-8ea69ab09e7f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:03.936508 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0bce2-4371-4425-b7a5-e1bf590f72d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:03.937123 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0bce2-4371-4425-b7a5-e1bf590f72d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:03.937405 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0bce2-4371-4425-b7a5-e1bf590f72d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:03.937679 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0bce2-4371-4425-b7a5-e1bf590f72d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:03.947152 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0bce2-4371-4425-b7a5-e1bf590f72d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:03.948592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0bce2-4371-4425-b7a5-e1bf590f72d2 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:03.959108 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1411/2827] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:04.059891 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2f558-b1b9-4910-9ff2-027966079802 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:04.069904 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2f558-b1b9-4910-9ff2-027966079802 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:04.074613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28ce3b4-3dc1-4692-aa6e-188fd2738228 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:04.075234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ce3b4-3dc1-4692-aa6e-188fd2738228 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:04.075524 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ce3b4-3dc1-4692-aa6e-188fd2738228 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:04.075794 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28ce3b4-3dc1-4692-aa6e-188fd2738228 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:04.098790 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:04.158935 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ce3b4-3dc1-4692-aa6e-188fd2738228 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:04.160517 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28ce3b4-3dc1-4692-aa6e-188fd2738228 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:04.170340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1417/2828] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:20:04.453994 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0bce2-4371-4425-b7a5-e1bf590f72d2 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:04.522392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0bce2-4371-4425-b7a5-e1bf590f72d2 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:04.526867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:04.527516 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:04.527760 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:04.528030 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] PATH_INFO: `/v3/users/8756b5633e8140b4977245cffa995ccb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:04.529155 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8756b5633e8140b4977245cffa995ccb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:04.531102 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:04.578501 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8756b5633e8140b4977245cffa995ccb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:04.585868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8756b5633e8140b4977245cffa995ccb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:04.586488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8756b5633e8140b4977245cffa995ccb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:04.587094 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8756b5633e8140b4977245cffa995ccb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:04.594759 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8756b5633e8140b4977245cffa995ccb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:04.615496 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1412/2829] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:20:04 2022] DELETE /identity/v3/users/8756b5633e8140b4977245cffa995ccb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:04.619914 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafc4b7-85ab-4b36-9b59-605352fff70c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:04.620190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafc4b7-85ab-4b36-9b59-605352fff70c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:04.620459 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bafc4b7-85ab-4b36-9b59-605352fff70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:04.639884 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4bafc4b7-85ab-4b36-9b59-605352fff70c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:04.641083 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4bafc4b7-85ab-4b36-9b59-605352fff70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:04.705340 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1418/2830] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 19 21:20:04.715599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:04.717080 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:04.732272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:04.779852 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afa95e-e44c-4c0c-9156-c77e191873c3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:04.783548 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc80b5-dfd9-49b9-bb48-ea4d013d8a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:04.784157 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc80b5-dfd9-49b9-bb48-ea4d013d8a7c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:04.784435 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc80b5-dfd9-49b9-bb48-ea4d013d8a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:04.784709 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc80b5-dfd9-49b9-bb48-ea4d013d8a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:04.794392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc80b5-dfd9-49b9-bb48-ea4d013d8a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:04.795900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc80b5-dfd9-49b9-bb48-ea4d013d8a7c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:04.805332 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1413/2831] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:05.098601 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafc4b7-85ab-4b36-9b59-605352fff70c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:05.100893 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4bafc4b7-85ab-4b36-9b59-605352fff70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:05.115968 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bafc4b7-85ab-4b36-9b59-605352fff70c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:05.124619 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:05.125268 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:05.125542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:05.125852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] PATH_INFO: `/v3/projects/f9dd808393454ae682a4e6f1425587f0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:05.127543 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9dd808393454ae682a4e6f1425587f0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:05.129869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:05.183275 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9dd808393454ae682a4e6f1425587f0', 'request_id': 'req-02983a02-27cb-4053-aa78-5b1892cbf185'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.193393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9dd808393454ae682a4e6f1425587f0', 'request_id': 'req-02983a02-27cb-4053-aa78-5b1892cbf185'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.200369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9dd808393454ae682a4e6f1425587f0', 'request_id': 'req-02983a02-27cb-4053-aa78-5b1892cbf185'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.211938 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] The token cache is being invalidate because project f9dd808393454ae682a4e6f1425587f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:20:05.212378 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.213146 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.214907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1419/2832] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:20:05 2022] DELETE /identity/v3/projects/f9dd808393454ae682a4e6f1425587f0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:05.218895 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc80b5-dfd9-49b9-bb48-ea4d013d8a7c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:05.230406 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc80b5-dfd9-49b9-bb48-ea4d013d8a7c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:05.304767 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:05.305421 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:05.305701 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:05.306010 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] PATH_INFO: `/v3/users/4eb92114e0a04f42b2872c0c3399ec5b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:05.307169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eb92114e0a04f42b2872c0c3399ec5b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:05.309267 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:05.364600 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eb92114e0a04f42b2872c0c3399ec5b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.373642 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eb92114e0a04f42b2872c0c3399ec5b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.374423 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eb92114e0a04f42b2872c0c3399ec5b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.375111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eb92114e0a04f42b2872c0c3399ec5b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.383058 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cc6ecfb0-8b5f-4153-96f2-69b4bbeac7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eb92114e0a04f42b2872c0c3399ec5b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.405495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1414/2833] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:20:05 2022] DELETE /identity/v3/users/4eb92114e0a04f42b2872c0c3399ec5b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:05.708160 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:05.709613 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:05.722704 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983a02-27cb-4053-aa78-5b1892cbf185 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:05.768618 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:05.786896 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:05.787531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:05.787809 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:05.788082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] PATH_INFO: `/v3/projects/a187d7870f4c4d84ae803a18729f428c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:05.789229 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a187d7870f4c4d84ae803a18729f428c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:05.791187 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:05.844010 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a187d7870f4c4d84ae803a18729f428c', 'request_id': 'req-348167cb-f4b3-49a3-8f85-3039cfa46001'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.850813 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a187d7870f4c4d84ae803a18729f428c', 'request_id': 'req-348167cb-f4b3-49a3-8f85-3039cfa46001'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.859309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a187d7870f4c4d84ae803a18729f428c', 'request_id': 'req-348167cb-f4b3-49a3-8f85-3039cfa46001'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.871520 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] The token cache is being invalidate because project a187d7870f4c4d84ae803a18729f428c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:20:05.871811 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.872477 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-348167cb-f4b3-49a3-8f85-3039cfa46001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:05.873990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1420/2834] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:20:05 2022] DELETE /identity/v3/projects/a187d7870f4c4d84ae803a18729f428c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:05.878801 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8f8b7-f967-46dd-a35b-a1101e7a94ee None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:05.879256 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8f8b7-f967-46dd-a35b-a1101e7a94ee None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:05.879353 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d8f8b7-f967-46dd-a35b-a1101e7a94ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:05.899712 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-88d8f8b7-f967-46dd-a35b-a1101e7a94ee None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:05.900934 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-88d8f8b7-f967-46dd-a35b-a1101e7a94ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:05.932728 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:05.966520 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1415/2835] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:05.974687 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a2e8d-ad64-4605-981f-33ff2af62448 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:05.974971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a2e8d-ad64-4605-981f-33ff2af62448 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:05.975243 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813a2e8d-ad64-4605-981f-33ff2af62448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:05.997341 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-813a2e8d-ad64-4605-981f-33ff2af62448 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:05.998584 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-813a2e8d-ad64-4605-981f-33ff2af62448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:06.006976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1421/2836] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:06.011351 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8f8b7-f967-46dd-a35b-a1101e7a94ee None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:06.023868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d8f8b7-f967-46dd-a35b-a1101e7a94ee None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:06.036900 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87f15d-985a-4e98-b5c7-caecd8418e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:06.037559 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87f15d-985a-4e98-b5c7-caecd8418e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:06.037868 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87f15d-985a-4e98-b5c7-caecd8418e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:06.038145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87f15d-985a-4e98-b5c7-caecd8418e82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:06.038806 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87f15d-985a-4e98-b5c7-caecd8418e82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:20:06.039114 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87f15d-985a-4e98-b5c7-caecd8418e82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:06.040732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87f15d-985a-4e98-b5c7-caecd8418e82 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:06.053947 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1416/2837] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:20:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 19 21:20:06.058269 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a2e8d-ad64-4605-981f-33ff2af62448 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:06.068182 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813a2e8d-ad64-4605-981f-33ff2af62448 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:06.072923 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78b53e-231a-4ca2-a8e3-761c05a81c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:06.073631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78b53e-231a-4ca2-a8e3-761c05a81c00 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:06.074001 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78b53e-231a-4ca2-a8e3-761c05a81c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:06.074934 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78b53e-231a-4ca2-a8e3-761c05a81c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:06.075698 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78b53e-231a-4ca2-a8e3-761c05a81c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:06.078074 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78b53e-231a-4ca2-a8e3-761c05a81c00 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:06.108729 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1422/2838] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:20:06 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:20:06.113816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87f15d-985a-4e98-b5c7-caecd8418e82 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:06.124120 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87f15d-985a-4e98-b5c7-caecd8418e82 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:06.182310 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e98582-6beb-4d10-aefc-40cad558bcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:06.182916 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98582-6beb-4d10-aefc-40cad558bcb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:06.183198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98582-6beb-4d10-aefc-40cad558bcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:06.183473 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e98582-6beb-4d10-aefc-40cad558bcb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:06.184106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e98582-6beb-4d10-aefc-40cad558bcb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:06.186044 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e98582-6beb-4d10-aefc-40cad558bcb1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:06.208355 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1417/2839] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:20:06 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:20:06.212649 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78b53e-231a-4ca2-a8e3-761c05a81c00 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:06.225251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78b53e-231a-4ca2-a8e3-761c05a81c00 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:06.230112 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d6a70-4558-420a-bda7-6ea90da3f5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:06.230971 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d6a70-4558-420a-bda7-6ea90da3f5f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:06.231286 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d6a70-4558-420a-bda7-6ea90da3f5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:06.231563 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d6a70-4558-420a-bda7-6ea90da3f5f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:06.232214 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d6a70-4558-420a-bda7-6ea90da3f5f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:06.233820 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2d6a70-4558-420a-bda7-6ea90da3f5f1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:06.246265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1423/2840] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:20:06.250448 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e98582-6beb-4d10-aefc-40cad558bcb1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:06.260531 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e98582-6beb-4d10-aefc-40cad558bcb1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:06.266007 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb4792-c295-4a7c-8e60-b11dc51dffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:06.266626 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb4792-c295-4a7c-8e60-b11dc51dffbd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:06.266908 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb4792-c295-4a7c-8e60-b11dc51dffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:06.267453 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb4792-c295-4a7c-8e60-b11dc51dffbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:06.268111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb4792-c295-4a7c-8e60-b11dc51dffbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:06.269941 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bb4792-c295-4a7c-8e60-b11dc51dffbd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:06.282156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1418/2841] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:20:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:20:06.286372 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d6a70-4558-420a-bda7-6ea90da3f5f1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:06.296469 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2d6a70-4558-420a-bda7-6ea90da3f5f1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:06.301316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a843c-7b4d-4fcc-b450-b47755c9a0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:06.302053 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a843c-7b4d-4fcc-b450-b47755c9a0ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:06.302353 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a843c-7b4d-4fcc-b450-b47755c9a0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:06.302610 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a843c-7b4d-4fcc-b450-b47755c9a0ba admin admin] PATH_INFO: `/v3/projects/256b96ccf7374edfa86d79ba2c478ca9/users/419f3322373d4fc4922cc3262ac30fca/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:06.318988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a843c-7b4d-4fcc-b450-b47755c9a0ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=256b96ccf7374edfa86d79ba2c478ca9, user_id=419f3322373d4fc4922cc3262ac30fca, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:06.321673 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a843c-7b4d-4fcc-b450-b47755c9a0ba admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:06.336471 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1424/2842] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:20:06 2022] PUT /identity/v3/projects/256b96ccf7374edfa86d79ba2c478ca9/users/419f3322373d4fc4922cc3262ac30fca/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:06.347391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ab228-b2c0-4f95-947c-5c77f4a065fd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:06.347676 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ab228-b2c0-4f95-947c-5c77f4a065fd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:06.347971 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ab228-b2c0-4f95-947c-5c77f4a065fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:06.377787 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1d4ab228-b2c0-4f95-947c-5c77f4a065fd None None] MFA Rules not processed for user `419f3322373d4fc4922cc3262ac30fca`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:06.379040 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1d4ab228-b2c0-4f95-947c-5c77f4a065fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:06.452997 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1419/2843] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:06.463995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a843c-7b4d-4fcc-b450-b47755c9a0ba admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:06.467039 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-564a843c-7b4d-4fcc-b450-b47755c9a0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:06.552611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a843c-7b4d-4fcc-b450-b47755c9a0ba admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:06.556330 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09295cc-fef4-4003-92cc-c799ef762311 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:06.556960 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09295cc-fef4-4003-92cc-c799ef762311 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:06.557241 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09295cc-fef4-4003-92cc-c799ef762311 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:06.557515 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09295cc-fef4-4003-92cc-c799ef762311 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:06.567134 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09295cc-fef4-4003-92cc-c799ef762311 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:06.568658 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09295cc-fef4-4003-92cc-c799ef762311 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:06.578654 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1425/2844] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:16.449648 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:16.573924 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:19.197855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ab228-b2c0-4f95-947c-5c77f4a065fd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:19.268816 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ab228-b2c0-4f95-947c-5c77f4a065fd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:19.275668 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:19.276376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:19.276670 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:19.276942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] PATH_INFO: `/v3/users/419f3322373d4fc4922cc3262ac30fca` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:19.278706 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=419f3322373d4fc4922cc3262ac30fca)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:19.280630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:19.336720 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '419f3322373d4fc4922cc3262ac30fca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:19.344935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '419f3322373d4fc4922cc3262ac30fca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:19.345613 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '419f3322373d4fc4922cc3262ac30fca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:19.346258 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '419f3322373d4fc4922cc3262ac30fca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:19.353284 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '419f3322373d4fc4922cc3262ac30fca'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:19.371430 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1420/2845] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:20:19 2022] DELETE /identity/v3/users/419f3322373d4fc4922cc3262ac30fca => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:19.375995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d25587-8322-463a-b05b-4bff45fe88ed None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:19.376532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d25587-8322-463a-b05b-4bff45fe88ed None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:19.376532 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d25587-8322-463a-b05b-4bff45fe88ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:19.395984 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-51d25587-8322-463a-b05b-4bff45fe88ed None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:19.397202 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-51d25587-8322-463a-b05b-4bff45fe88ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:19.459224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1426/2846] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:19.469022 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:19.470480 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:19.532387 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c81ca-4b63-46c1-b539-736cc78f3d6c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:19.536825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3a1e7-5fb3-499c-ad3e-074fc016aa00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:19.537467 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3a1e7-5fb3-499c-ad3e-074fc016aa00 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:19.537749 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3a1e7-5fb3-499c-ad3e-074fc016aa00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:19.538039 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3a1e7-5fb3-499c-ad3e-074fc016aa00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:19.549403 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3a1e7-5fb3-499c-ad3e-074fc016aa00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:19.550967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3a1e7-5fb3-499c-ad3e-074fc016aa00 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:19.563138 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1421/2847] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:19.847611 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d25587-8322-463a-b05b-4bff45fe88ed None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:19.849222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-51d25587-8322-463a-b05b-4bff45fe88ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:19.864988 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d25587-8322-463a-b05b-4bff45fe88ed None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:19.870593 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:19.871678 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:19.872024 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:19.872424 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] PATH_INFO: `/v3/projects/256b96ccf7374edfa86d79ba2c478ca9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:19.873651 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=256b96ccf7374edfa86d79ba2c478ca9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:19.875608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:19.924753 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '256b96ccf7374edfa86d79ba2c478ca9', 'request_id': 'req-79659bff-c88f-4d2a-96c6-ade96f238884'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:19.931854 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '256b96ccf7374edfa86d79ba2c478ca9', 'request_id': 'req-79659bff-c88f-4d2a-96c6-ade96f238884'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:19.938151 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '256b96ccf7374edfa86d79ba2c478ca9', 'request_id': 'req-79659bff-c88f-4d2a-96c6-ade96f238884'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:19.948936 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] The token cache is being invalidate because project 256b96ccf7374edfa86d79ba2c478ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:20:19.949917 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:19.950791 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79659bff-c88f-4d2a-96c6-ade96f238884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:19.952278 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1427/2848] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:20:19 2022] DELETE /identity/v3/projects/256b96ccf7374edfa86d79ba2c478ca9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:19.957099 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7780298-fa18-4549-b20f-6ea13b1a2fa3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:19.957382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7780298-fa18-4549-b20f-6ea13b1a2fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:19.957652 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7780298-fa18-4549-b20f-6ea13b1a2fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:19.978270 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e7780298-fa18-4549-b20f-6ea13b1a2fa3 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:19.979468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e7780298-fa18-4549-b20f-6ea13b1a2fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:19.997361 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:20.042569 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1422/2849] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:20.051176 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0df44-e82e-419e-b3f7-3b903ebcce8e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:20.051461 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0df44-e82e-419e-b3f7-3b903ebcce8e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:20.051735 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b0df44-e82e-419e-b3f7-3b903ebcce8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:20.072012 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-36b0df44-e82e-419e-b3f7-3b903ebcce8e None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:20.073222 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-36b0df44-e82e-419e-b3f7-3b903ebcce8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:20.081172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1428/2850] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:20.085523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7780298-fa18-4549-b20f-6ea13b1a2fa3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:20.096641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7780298-fa18-4549-b20f-6ea13b1a2fa3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:20.101290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8df2e-ef1e-4d31-bbd3-6f164c620776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:20.101942 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8df2e-ef1e-4d31-bbd3-6f164c620776 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:20.102245 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8df2e-ef1e-4d31-bbd3-6f164c620776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:20.102514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d8df2e-ef1e-4d31-bbd3-6f164c620776 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:20.103156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8df2e-ef1e-4d31-bbd3-6f164c620776 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:20:20.103459 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8df2e-ef1e-4d31-bbd3-6f164c620776 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:20.105050 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d8df2e-ef1e-4d31-bbd3-6f164c620776 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:20.117350 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1423/2851] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:20:20.122359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0df44-e82e-419e-b3f7-3b903ebcce8e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:20.131684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b0df44-e82e-419e-b3f7-3b903ebcce8e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:20.137000 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8c63f-b433-4f6f-8016-6321fd16d466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:20.137650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8c63f-b433-4f6f-8016-6321fd16d466 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:20.137958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8c63f-b433-4f6f-8016-6321fd16d466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:20.138228 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e8c63f-b433-4f6f-8016-6321fd16d466 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:20.138846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8c63f-b433-4f6f-8016-6321fd16d466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:20.140734 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e8c63f-b433-4f6f-8016-6321fd16d466 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:20.167723 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1429/2852] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:20:20 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:20:20.172158 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8df2e-ef1e-4d31-bbd3-6f164c620776 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:20.182990 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d8df2e-ef1e-4d31-bbd3-6f164c620776 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:20.240077 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ba037-8355-409e-b80a-8b24a133adb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:20.240744 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ba037-8355-409e-b80a-8b24a133adb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:20.240948 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ba037-8355-409e-b80a-8b24a133adb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:20.241223 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ba037-8355-409e-b80a-8b24a133adb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:20.241878 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ba037-8355-409e-b80a-8b24a133adb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:20.243939 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ba037-8355-409e-b80a-8b24a133adb1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:20.266063 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1424/2853] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:20:20 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:20:20.270662 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8c63f-b433-4f6f-8016-6321fd16d466 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:20.281866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e8c63f-b433-4f6f-8016-6321fd16d466 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:20.286527 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752387d-e8ce-4534-8de5-93d7974dcd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:20.287117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752387d-e8ce-4534-8de5-93d7974dcd99 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:20.287391 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752387d-e8ce-4534-8de5-93d7974dcd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:20.287666 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752387d-e8ce-4534-8de5-93d7974dcd99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:20.288285 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752387d-e8ce-4534-8de5-93d7974dcd99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:20.289907 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5752387d-e8ce-4534-8de5-93d7974dcd99 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:20.302485 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1430/2854] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:20:20.306865 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ba037-8355-409e-b80a-8b24a133adb1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:20.317408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ba037-8355-409e-b80a-8b24a133adb1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:20.322041 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60121c-6809-4608-97e8-f264a440ea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:20.322633 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60121c-6809-4608-97e8-f264a440ea37 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:20.322928 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60121c-6809-4608-97e8-f264a440ea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:20.323182 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60121c-6809-4608-97e8-f264a440ea37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:20.323818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60121c-6809-4608-97e8-f264a440ea37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:20.326368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60121c-6809-4608-97e8-f264a440ea37 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:20.336809 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1425/2855] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 19 21:20:20.342079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752387d-e8ce-4534-8de5-93d7974dcd99 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:20.354667 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752387d-e8ce-4534-8de5-93d7974dcd99 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:20.359716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9973aa2-b139-42a7-8c52-bd1f8b569ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:20.360362 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9973aa2-b139-42a7-8c52-bd1f8b569ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:20.360636 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9973aa2-b139-42a7-8c52-bd1f8b569ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:20.360905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9973aa2-b139-42a7-8c52-bd1f8b569ffc admin admin] PATH_INFO: `/v3/projects/b25b6765de1844b3badd06ff0a19412d/users/b6db6d6fa01a43cea7934dc8ccef0e5a/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:20.375733 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9973aa2-b139-42a7-8c52-bd1f8b569ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b25b6765de1844b3badd06ff0a19412d, user_id=b6db6d6fa01a43cea7934dc8ccef0e5a, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:20.378404 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9973aa2-b139-42a7-8c52-bd1f8b569ffc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:20.392720 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1431/2856] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:20:20 2022] PUT /identity/v3/projects/b25b6765de1844b3badd06ff0a19412d/users/b6db6d6fa01a43cea7934dc8ccef0e5a/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:20.403362 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f930f-08e1-42ae-bccf-cc6157e759be None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:20.403643 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f930f-08e1-42ae-bccf-cc6157e759be None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:20.403918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f930f-08e1-42ae-bccf-cc6157e759be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:20.431544 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-230f930f-08e1-42ae-bccf-cc6157e759be None None] MFA Rules not processed for user `b6db6d6fa01a43cea7934dc8ccef0e5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:20.432778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-230f930f-08e1-42ae-bccf-cc6157e759be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:20.497465 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1426/2857] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:20.507973 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9973aa2-b139-42a7-8c52-bd1f8b569ffc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:20.509670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e9973aa2-b139-42a7-8c52-bd1f8b569ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:20.533312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:20.571545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9973aa2-b139-42a7-8c52-bd1f8b569ffc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:20.575320 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d23d9d-76a3-49bb-a03c-f13497a50055 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:20.575928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d23d9d-76a3-49bb-a03c-f13497a50055 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:20.576205 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d23d9d-76a3-49bb-a03c-f13497a50055 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:20.576472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d23d9d-76a3-49bb-a03c-f13497a50055 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:20.584710 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d23d9d-76a3-49bb-a03c-f13497a50055 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:20.586171 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d23d9d-76a3-49bb-a03c-f13497a50055 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:20.595599 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1432/2858] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:21.098797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f930f-08e1-42ae-bccf-cc6157e759be None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:21.100212 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-230f930f-08e1-42ae-bccf-cc6157e759be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:21.120509 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:21.160716 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f930f-08e1-42ae-bccf-cc6157e759be None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:21.164427 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25801152-1a23-4019-82f4-e911f6f09c4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:21.164936 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25801152-1a23-4019-82f4-e911f6f09c4d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:21.165214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25801152-1a23-4019-82f4-e911f6f09c4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:21.165490 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25801152-1a23-4019-82f4-e911f6f09c4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:21.167241 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-25801152-1a23-4019-82f4-e911f6f09c4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:21.230950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25801152-1a23-4019-82f4-e911f6f09c4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:21.232392 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25801152-1a23-4019-82f4-e911f6f09c4d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:21.241922 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1427/2859] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:22.879585 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d23d9d-76a3-49bb-a03c-f13497a50055 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:22.881192 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-41d23d9d-76a3-49bb-a03c-f13497a50055 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:22.984418 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d23d9d-76a3-49bb-a03c-f13497a50055 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:22.990484 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b178080-de4e-4eb3-97db-6d22452243a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:22.991267 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b178080-de4e-4eb3-97db-6d22452243a4 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:22.991631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b178080-de4e-4eb3-97db-6d22452243a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:22.991976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b178080-de4e-4eb3-97db-6d22452243a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:22.994008 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3b178080-de4e-4eb3-97db-6d22452243a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:23.007802 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b178080-de4e-4eb3-97db-6d22452243a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:23.009408 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b178080-de4e-4eb3-97db-6d22452243a4 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:23.021868 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1433/2860] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:31.237723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=42 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:33.016755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=51 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:34.962844 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25801152-1a23-4019-82f4-e911f6f09c4d service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:35.025125 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25801152-1a23-4019-82f4-e911f6f09c4d service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:35.029918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:35.030589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:35.030867 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:35.031140 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] PATH_INFO: `/v3/users/b6db6d6fa01a43cea7934dc8ccef0e5a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:35.032320 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6db6d6fa01a43cea7934dc8ccef0e5a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:35.034276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:35.080607 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6db6d6fa01a43cea7934dc8ccef0e5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:35.088378 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6db6d6fa01a43cea7934dc8ccef0e5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:35.089060 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6db6d6fa01a43cea7934dc8ccef0e5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:35.090027 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6db6d6fa01a43cea7934dc8ccef0e5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:35.096876 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6db6d6fa01a43cea7934dc8ccef0e5a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:35.113592 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1428/2861] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:20:34 2022] DELETE /identity/v3/users/b6db6d6fa01a43cea7934dc8ccef0e5a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:35.118087 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180a79a-0ec8-4f5a-b410-c1c67a630730 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:35.118556 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180a79a-0ec8-4f5a-b410-c1c67a630730 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:35.118660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2180a79a-0ec8-4f5a-b410-c1c67a630730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:35.139670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2180a79a-0ec8-4f5a-b410-c1c67a630730 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:35.141296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2180a79a-0ec8-4f5a-b410-c1c67a630730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:35.201529 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1434/2862] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:35.211555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:35.213000 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:35.273991 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f15ffb-94ac-4837-88ef-1f283850da66 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:35.277619 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44facf66-3686-4ae3-ab02-32d8da60c080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:35.278247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44facf66-3686-4ae3-ab02-32d8da60c080 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:35.278523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44facf66-3686-4ae3-ab02-32d8da60c080 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:35.278792 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44facf66-3686-4ae3-ab02-32d8da60c080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:35.288390 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44facf66-3686-4ae3-ab02-32d8da60c080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:35.289833 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44facf66-3686-4ae3-ab02-32d8da60c080 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:35.300340 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1429/2863] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:35.582376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180a79a-0ec8-4f5a-b410-c1c67a630730 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:35.583861 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2180a79a-0ec8-4f5a-b410-c1c67a630730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:35.595530 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2180a79a-0ec8-4f5a-b410-c1c67a630730 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:35.600443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:35.601079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:35.601356 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:35.601630 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] PATH_INFO: `/v3/projects/b25b6765de1844b3badd06ff0a19412d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:35.602976 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b25b6765de1844b3badd06ff0a19412d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:35.604846 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:35.654581 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b25b6765de1844b3badd06ff0a19412d', 'request_id': 'req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:35.662184 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b25b6765de1844b3badd06ff0a19412d', 'request_id': 'req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:35.669130 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b25b6765de1844b3badd06ff0a19412d', 'request_id': 'req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:35.681848 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] The token cache is being invalidate because project b25b6765de1844b3badd06ff0a19412d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:20:35.682140 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:35.682856 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5c9ea05d-6653-4876-bcd8-cba3a9c3dcc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:35.684388 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1435/2864] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:20:35 2022] DELETE /identity/v3/projects/b25b6765de1844b3badd06ff0a19412d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:35.690370 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6314c2-0078-49e3-b8fb-5487a2c7155c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:35.690653 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6314c2-0078-49e3-b8fb-5487a2c7155c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:35.690927 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6314c2-0078-49e3-b8fb-5487a2c7155c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:35.711645 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7b6314c2-0078-49e3-b8fb-5487a2c7155c None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:35.712926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7b6314c2-0078-49e3-b8fb-5487a2c7155c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:35.733391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:35.787495 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1430/2865] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:35.796392 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf21a1-8a17-4ba8-aba0-1ba919751dd3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:35.796695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf21a1-8a17-4ba8-aba0-1ba919751dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:35.796965 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf21a1-8a17-4ba8-aba0-1ba919751dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:35.818293 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-45cf21a1-8a17-4ba8-aba0-1ba919751dd3 None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:35.819518 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-45cf21a1-8a17-4ba8-aba0-1ba919751dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:35.828542 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1436/2866] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:35.833246 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6314c2-0078-49e3-b8fb-5487a2c7155c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:35.843300 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6314c2-0078-49e3-b8fb-5487a2c7155c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:35.847834 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ee8c1-23f8-49a5-abd5-05370a8da44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:35.848425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ee8c1-23f8-49a5-abd5-05370a8da44c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:35.848778 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ee8c1-23f8-49a5-abd5-05370a8da44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:35.848955 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ee8c1-23f8-49a5-abd5-05370a8da44c admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:35.849597 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ee8c1-23f8-49a5-abd5-05370a8da44c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 19 21:20:35.849932 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ee8c1-23f8-49a5-abd5-05370a8da44c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:35.851530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ee8c1-23f8-49a5-abd5-05370a8da44c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:35.864828 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1431/2867] 10.222.0.16 () {58 vars in 1153 bytes} [Fri Aug 19 21:20:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 19 21:20:35.869359 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf21a1-8a17-4ba8-aba0-1ba919751dd3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:35.880009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf21a1-8a17-4ba8-aba0-1ba919751dd3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:35.884906 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2e8d-3c27-4d0e-9502-8c0240383e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:35.885531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2e8d-3c27-4d0e-9502-8c0240383e63 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:35.885837 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2e8d-3c27-4d0e-9502-8c0240383e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:35.886114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d2e8d-3c27-4d0e-9502-8c0240383e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:35.886768 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d2e8d-3c27-4d0e-9502-8c0240383e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:35.888736 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d2e8d-3c27-4d0e-9502-8c0240383e63 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:35.919719 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1437/2868] 10.222.0.16 () {60 vars in 1153 bytes} [Fri Aug 19 21:20:35 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:20:35.925043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ee8c1-23f8-49a5-abd5-05370a8da44c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:35.935673 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ee8c1-23f8-49a5-abd5-05370a8da44c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:35.996122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6253e18-551b-46d2-bea6-029eee1b9ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:35.996786 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6253e18-551b-46d2-bea6-029eee1b9ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:35.996904 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6253e18-551b-46d2-bea6-029eee1b9ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:35.997176 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6253e18-551b-46d2-bea6-029eee1b9ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:35.997832 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6253e18-551b-46d2-bea6-029eee1b9ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:35.999950 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6253e18-551b-46d2-bea6-029eee1b9ba2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:36.021349 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1432/2869] 10.222.0.16 () {60 vars in 1144 bytes} [Fri Aug 19 21:20:35 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 19 21:20:36.025816 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2e8d-3c27-4d0e-9502-8c0240383e63 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:36.036954 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d2e8d-3c27-4d0e-9502-8c0240383e63 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:36.041869 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3645a-0789-48eb-98c8-9ea26a9a3a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:36.042472 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3645a-0789-48eb-98c8-9ea26a9a3a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:36.042752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3645a-0789-48eb-98c8-9ea26a9a3a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:36.043029 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3645a-0789-48eb-98c8-9ea26a9a3a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:36.043669 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3645a-0789-48eb-98c8-9ea26a9a3a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:36.045916 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3645a-0789-48eb-98c8-9ea26a9a3a43 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:36.055948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1438/2870] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:20:36.060181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6253e18-551b-46d2-bea6-029eee1b9ba2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:36.070357 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6253e18-551b-46d2-bea6-029eee1b9ba2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:36.075043 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ee15c-effc-4bba-9a33-07c5c3f9918b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:36.075636 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ee15c-effc-4bba-9a33-07c5c3f9918b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:36.075913 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ee15c-effc-4bba-9a33-07c5c3f9918b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:36.076186 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ee15c-effc-4bba-9a33-07c5c3f9918b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:36.077497 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ee15c-effc-4bba-9a33-07c5c3f9918b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:36.079133 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ee15c-effc-4bba-9a33-07c5c3f9918b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:36.090698 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1433/2871] 10.222.0.16 () {58 vars in 1122 bytes} [Fri Aug 19 21:20:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 19 21:20:36.095824 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3645a-0789-48eb-98c8-9ea26a9a3a43 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:36.105888 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3645a-0789-48eb-98c8-9ea26a9a3a43 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:36.110731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a7dc1-aeef-42b1-9e87-f0acdf1cc97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:36.111369 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7dc1-aeef-42b1-9e87-f0acdf1cc97c admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:36.111653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7dc1-aeef-42b1-9e87-f0acdf1cc97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:36.111931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7dc1-aeef-42b1-9e87-f0acdf1cc97c admin admin] PATH_INFO: `/v3/projects/c290ef5d11bd457faab2536a334b9bb7/users/d96315b58c9e4ed38c97c0d3b50e8069/roles/f8c927860d6941559283c83fcaaf93d8` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:36.126785 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7dc1-aeef-42b1-9e87-f0acdf1cc97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c290ef5d11bd457faab2536a334b9bb7, user_id=d96315b58c9e4ed38c97c0d3b50e8069, role_id=f8c927860d6941559283c83fcaaf93d8)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:36.129486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7dc1-aeef-42b1-9e87-f0acdf1cc97c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:36.147295 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1439/2872] 10.222.0.16 () {60 vars in 1483 bytes} [Fri Aug 19 21:20:36 2022] PUT /identity/v3/projects/c290ef5d11bd457faab2536a334b9bb7/users/d96315b58c9e4ed38c97c0d3b50e8069/roles/f8c927860d6941559283c83fcaaf93d8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:36.152732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139054e-fdbd-46dd-95af-7727ce39932a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:36.153009 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139054e-fdbd-46dd-95af-7727ce39932a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:36.153281 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5139054e-fdbd-46dd-95af-7727ce39932a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:36.172879 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5139054e-fdbd-46dd-95af-7727ce39932a None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:36.174127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5139054e-fdbd-46dd-95af-7727ce39932a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:36.234475 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1434/2873] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:36.244693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a7dc1-aeef-42b1-9e87-f0acdf1cc97c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:36.246975 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ff3a7dc1-aeef-42b1-9e87-f0acdf1cc97c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:36.269386 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:36.306065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a7dc1-aeef-42b1-9e87-f0acdf1cc97c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:36.309699 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bc062-2d8d-4cf2-a3ba-50202678da60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:36.310322 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bc062-2d8d-4cf2-a3ba-50202678da60 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:36.310594 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bc062-2d8d-4cf2-a3ba-50202678da60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:36.311044 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77bc062-2d8d-4cf2-a3ba-50202678da60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:36.319784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bc062-2d8d-4cf2-a3ba-50202678da60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:36.321190 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77bc062-2d8d-4cf2-a3ba-50202678da60 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:36.330966 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1440/2874] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:36.986434 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eb720-1509-4693-99d5-c3dce7ca4144 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:36.986725 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eb720-1509-4693-99d5-c3dce7ca4144 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:36.987002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730eb720-1509-4693-99d5-c3dce7ca4144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:37.015658 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-730eb720-1509-4693-99d5-c3dce7ca4144 None None] MFA Rules not processed for user `d96315b58c9e4ed38c97c0d3b50e8069`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:37.016881 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-730eb720-1509-4693-99d5-c3dce7ca4144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:37.036883 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:37.100762 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1435/2875] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:37.111707 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bc062-2d8d-4cf2-a3ba-50202678da60 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:37.113107 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c77bc062-2d8d-4cf2-a3ba-50202678da60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:37.176309 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77bc062-2d8d-4cf2-a3ba-50202678da60 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:37.180161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41f4cb-48bc-4e39-a6c7-79e53b1a0b12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:37.180766 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41f4cb-48bc-4e39-a6c7-79e53b1a0b12 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:37.181041 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41f4cb-48bc-4e39-a6c7-79e53b1a0b12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:37.181308 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41f4cb-48bc-4e39-a6c7-79e53b1a0b12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:37.191082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41f4cb-48bc-4e39-a6c7-79e53b1a0b12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:37.192523 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41f4cb-48bc-4e39-a6c7-79e53b1a0b12 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:37.202676 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1441/2876] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:37.714129 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730eb720-1509-4693-99d5-c3dce7ca4144 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:37.715582 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-730eb720-1509-4693-99d5-c3dce7ca4144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:37.729264 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730eb720-1509-4693-99d5-c3dce7ca4144 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:37.734661 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:37.735299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:37.735581 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:37.735855 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] PATH_INFO: `/v3/users/dc6e32aa37f445d1a74df1b2dd98aa2f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:37.737011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6e32aa37f445d1a74df1b2dd98aa2f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:37.738980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:37.787515 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6e32aa37f445d1a74df1b2dd98aa2f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:37.794946 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6e32aa37f445d1a74df1b2dd98aa2f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:37.796479 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6e32aa37f445d1a74df1b2dd98aa2f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:37.797169 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6e32aa37f445d1a74df1b2dd98aa2f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:37.804164 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6e32aa37f445d1a74df1b2dd98aa2f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:37.821625 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1436/2877] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:20:37 2022] DELETE /identity/v3/users/dc6e32aa37f445d1a74df1b2dd98aa2f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:37.826098 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d035df0-2fbd-4091-88b4-9af1a702e8fb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:37.826390 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d035df0-2fbd-4091-88b4-9af1a702e8fb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:37.826668 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d035df0-2fbd-4091-88b4-9af1a702e8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:37.845754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4d035df0-2fbd-4091-88b4-9af1a702e8fb None None] MFA Rules not processed for user `3476c91ad03941919217e0940471d201`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:37.846970 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4d035df0-2fbd-4091-88b4-9af1a702e8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:37.907577 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1442/2878] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:37.917400 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:37.918829 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:37.978918 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed656d8-f83b-480d-b504-28bfb6405ec3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:37.982529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c38638-2c4b-40ae-870f-4b92455b9a69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:37.983821 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c38638-2c4b-40ae-870f-4b92455b9a69 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:37.984092 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c38638-2c4b-40ae-870f-4b92455b9a69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:37.984358 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c38638-2c4b-40ae-870f-4b92455b9a69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:37.993513 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c38638-2c4b-40ae-870f-4b92455b9a69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:37.994976 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c38638-2c4b-40ae-870f-4b92455b9a69 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:38.005145 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1437/2879] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:38.280273 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d035df0-2fbd-4091-88b4-9af1a702e8fb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:38.282082 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4d035df0-2fbd-4091-88b4-9af1a702e8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:38.298003 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d035df0-2fbd-4091-88b4-9af1a702e8fb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:38.303422 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:38.304065 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:38.304316 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:38.304589 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] PATH_INFO: `/v3/projects/d6295d3980694efd879eff5e64d848b2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:38.305990 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6295d3980694efd879eff5e64d848b2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:38.307836 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:38.360419 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6295d3980694efd879eff5e64d848b2', 'request_id': 'req-f29fcf3a-75b4-4512-8377-997c696759da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:38.368096 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6295d3980694efd879eff5e64d848b2', 'request_id': 'req-f29fcf3a-75b4-4512-8377-997c696759da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:38.372048 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:38.375208 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6295d3980694efd879eff5e64d848b2', 'request_id': 'req-f29fcf3a-75b4-4512-8377-997c696759da'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:38.387744 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] The token cache is being invalidate because project d6295d3980694efd879eff5e64d848b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:20:38.388035 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:38.388684 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:20:38.390317 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1443/2880] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:20:38 2022] DELETE /identity/v3/projects/d6295d3980694efd879eff5e64d848b2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:20:38.581641 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad891652-ac81-4f42-ab9b-a73b73061ed0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:38.582247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad891652-ac81-4f42-ab9b-a73b73061ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:38.582247 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad891652-ac81-4f42-ab9b-a73b73061ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:38.604137 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ad891652-ac81-4f42-ab9b-a73b73061ed0 None None] MFA Rules not processed for user `d96315b58c9e4ed38c97c0d3b50e8069`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:20:38.605391 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ad891652-ac81-4f42-ab9b-a73b73061ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:38.636807 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:38.665555 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1438/2881] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:20:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:20:38.903679 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:20:38.905105 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:38.969094 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29fcf3a-75b4-4512-8377-997c696759da admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:20:38.972686 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b725d-f54b-4362-bbc4-75da80f21fe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:20:38.973296 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b725d-f54b-4362-bbc4-75da80f21fe7 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:20:38.973575 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b725d-f54b-4362-bbc4-75da80f21fe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:20:38.973886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b725d-f54b-4362-bbc4-75da80f21fe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:20:38.975683 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-228b725d-f54b-4362-bbc4-75da80f21fe7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:20:38.986110 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b725d-f54b-4362-bbc4-75da80f21fe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:20:38.987716 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b725d-f54b-4362-bbc4-75da80f21fe7 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:20:38.996152 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1444/2882] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:20:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:20:48.658899 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=10 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:20:48.992351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=1,keystone:SELECT=30 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:21:26.484591 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad891652-ac81-4f42-ab9b-a73b73061ed0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:21:26.486419 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ad891652-ac81-4f42-ab9b-a73b73061ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:21:26.571983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad891652-ac81-4f42-ab9b-a73b73061ed0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:21:26.577393 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9446217-787c-4054-9787-3e8bd9f5ef0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:21:26.578214 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9446217-787c-4054-9787-3e8bd9f5ef0b service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:21:26.579026 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9446217-787c-4054-9787-3e8bd9f5ef0b service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:21:26.579368 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9446217-787c-4054-9787-3e8bd9f5ef0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:21:26.581251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e9446217-787c-4054-9787-3e8bd9f5ef0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:21:26.594887 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9446217-787c-4054-9787-3e8bd9f5ef0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:21:26.596746 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9446217-787c-4054-9787-3e8bd9f5ef0b service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:21:26.607911 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1439/2883] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:21:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:21:36.603179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=32 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:21:56.976491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbf0ef-a5cb-4127-bb96-d570b0b508e8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:21:56.977064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbf0ef-a5cb-4127-bb96-d570b0b508e8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:21:56.977064 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcbf0ef-a5cb-4127-bb96-d570b0b508e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:21:56.996420 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-adcbf0ef-a5cb-4127-bb96-d570b0b508e8 None None] MFA Rules not processed for user `d96315b58c9e4ed38c97c0d3b50e8069`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:21:56.997652 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-adcbf0ef-a5cb-4127-bb96-d570b0b508e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:21:57.004660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1445/2884] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:21:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:21:57.254073 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9446217-787c-4054-9787-3e8bd9f5ef0b service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:21:57.269183 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9446217-787c-4054-9787-3e8bd9f5ef0b service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:21:57.274106 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db960eb-102f-408e-9d8e-2f48c3114b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:21:57.274715 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db960eb-102f-408e-9d8e-2f48c3114b61 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:21:57.275127 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db960eb-102f-408e-9d8e-2f48c3114b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:21:57.275496 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db960eb-102f-408e-9d8e-2f48c3114b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:21:57.277346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4db960eb-102f-408e-9d8e-2f48c3114b61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:21:57.287192 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db960eb-102f-408e-9d8e-2f48c3114b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:21:57.288614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db960eb-102f-408e-9d8e-2f48c3114b61 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:21:57.297286 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1440/2885] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:21:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:22:06.991050 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:22:07.293526 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:22:12.886375 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbf0ef-a5cb-4127-bb96-d570b0b508e8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:22:12.888172 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-adcbf0ef-a5cb-4127-bb96-d570b0b508e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:22:12.901133 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcbf0ef-a5cb-4127-bb96-d570b0b508e8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:22:12.904781 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9a2eb-8965-44aa-a3c1-69afec271217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:22:12.905366 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9a2eb-8965-44aa-a3c1-69afec271217 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:22:12.905644 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9a2eb-8965-44aa-a3c1-69afec271217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:22:12.905948 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab9a2eb-8965-44aa-a3c1-69afec271217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:22:12.907803 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7ab9a2eb-8965-44aa-a3c1-69afec271217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:22:12.967334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9a2eb-8965-44aa-a3c1-69afec271217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:22:12.968661 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab9a2eb-8965-44aa-a3c1-69afec271217 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:22:12.977797 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1446/2886] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:22:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:22:20.157540 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db960eb-102f-408e-9d8e-2f48c3114b61 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:22:20.166597 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db960eb-102f-408e-9d8e-2f48c3114b61 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:22:20.170295 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb852e6-be84-424c-b57f-233a990228c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:22:20.170893 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb852e6-be84-424c-b57f-233a990228c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:22:20.171170 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb852e6-be84-424c-b57f-233a990228c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:22:20.171444 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb852e6-be84-424c-b57f-233a990228c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:22:20.173198 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fbb852e6-be84-424c-b57f-233a990228c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:22:20.236988 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb852e6-be84-424c-b57f-233a990228c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:22:20.238474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb852e6-be84-424c-b57f-233a990228c5 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:22:20.249436 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1441/2887] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:22:22.973545 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=28 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:22:23.572089 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9a2eb-8965-44aa-a3c1-69afec271217 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:22:23.588052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab9a2eb-8965-44aa-a3c1-69afec271217 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:22:23.594244 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee6d86-a269-4430-b700-362866e29516 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:22:23.595139 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee6d86-a269-4430-b700-362866e29516 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:22:23.595623 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee6d86-a269-4430-b700-362866e29516 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:22:23.596102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee6d86-a269-4430-b700-362866e29516 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:22:23.599136 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e6ee6d86-a269-4430-b700-362866e29516 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:22:23.614195 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee6d86-a269-4430-b700-362866e29516 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:22:23.615903 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee6d86-a269-4430-b700-362866e29516 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:22:23.630102 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1447/2888] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:22:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:22:26.845664 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb852e6-be84-424c-b57f-233a990228c5 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:22:26.847299 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fbb852e6-be84-424c-b57f-233a990228c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:22:26.862512 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb852e6-be84-424c-b57f-233a990228c5 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:22:26.868028 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:22:26.868674 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:22:26.868954 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:22:26.869231 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] PATH_INFO: `/v3/users/f7ed8b3dc88c4fadbb0839fd848397d1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:22:26.871506 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7ed8b3dc88c4fadbb0839fd848397d1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:22:26.873681 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:22:26.924905 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7ed8b3dc88c4fadbb0839fd848397d1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:26.936179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7ed8b3dc88c4fadbb0839fd848397d1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:26.936817 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7ed8b3dc88c4fadbb0839fd848397d1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:26.937443 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7ed8b3dc88c4fadbb0839fd848397d1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:26.944742 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7ed8b3dc88c4fadbb0839fd848397d1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:26.964474 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1442/2889] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:22:26 2022] DELETE /identity/v3/users/f7ed8b3dc88c4fadbb0839fd848397d1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:22:27.156364 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee6d86-a269-4430-b700-362866e29516 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:22:27.177925 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee6d86-a269-4430-b700-362866e29516 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:22:27.243558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:22:27.244189 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:22:27.245066 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:22:27.245952 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] PATH_INFO: `/v3/projects/b8d769840e964ef5923aa9c28265be03` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:22:27.247093 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8d769840e964ef5923aa9c28265be03)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:22:27.249026 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:22:27.301121 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8d769840e964ef5923aa9c28265be03', 'request_id': 'req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.312638 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8d769840e964ef5923aa9c28265be03', 'request_id': 'req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.318770 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8d769840e964ef5923aa9c28265be03', 'request_id': 'req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.330784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] The token cache is being invalidate because project b8d769840e964ef5923aa9c28265be03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:22:27.331079 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.331732 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.333215 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1448/2890] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:22:27 2022] DELETE /identity/v3/projects/b8d769840e964ef5923aa9c28265be03 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:22:27.337290 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:22:27.338818 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:22:27.352416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beb4470-0f51-4da6-92b6-f6b692147b9a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:22:27.413384 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:22:27.414061 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:22:27.414341 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:22:27.414615 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] PATH_INFO: `/v3/users/b329612022194c9dacf15ec93814fcfd` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:22:27.415779 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b329612022194c9dacf15ec93814fcfd)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:22:27.418655 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:22:27.425181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:22:27.472011 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b329612022194c9dacf15ec93814fcfd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.479251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b329612022194c9dacf15ec93814fcfd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.479935 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b329612022194c9dacf15ec93814fcfd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.480578 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b329612022194c9dacf15ec93814fcfd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.487805 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b329612022194c9dacf15ec93814fcfd'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.506514 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1443/2891] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:22:27 2022] DELETE /identity/v3/users/b329612022194c9dacf15ec93814fcfd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:22:27.686600 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:22:27.688117 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:22:27.702843 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a44e8af-80b6-4fa3-996b-5eb3eaaa1af6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:22:27.773005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:22:27.774019 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:22:27.774693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:22:27.774933 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] PATH_INFO: `/v3/projects/3b027eba284e4e6a9d6b018c23a0fd10` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:22:27.778379 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b027eba284e4e6a9d6b018c23a0fd10)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:22:27.780548 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:22:27.809922 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:22:27.847034 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b027eba284e4e6a9d6b018c23a0fd10', 'request_id': 'req-198d44e2-5a55-440e-8138-77c6beab21e6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.857875 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b027eba284e4e6a9d6b018c23a0fd10', 'request_id': 'req-198d44e2-5a55-440e-8138-77c6beab21e6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.864693 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b027eba284e4e6a9d6b018c23a0fd10', 'request_id': 'req-198d44e2-5a55-440e-8138-77c6beab21e6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.877258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] The token cache is being invalidate because project 3b027eba284e4e6a9d6b018c23a0fd10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:22:27.877549 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.878231 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:22:27.879928 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1449/2892] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:22:27 2022] DELETE /identity/v3/projects/3b027eba284e4e6a9d6b018c23a0fd10 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:22:37.503098 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:22:37.874009 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:23:00.785326 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:23:00.787037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:23:00.845692 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:23:00.847752 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:23:00.865366 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672fca01-3c7f-444b-b663-adcacd079213 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:23:00.920852 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d44e2-5a55-440e-8138-77c6beab21e6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:23:00.925546 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:23:00.926210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:23:00.926491 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:23:00.926767 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] PATH_INFO: `/v3/users/d96315b58c9e4ed38c97c0d3b50e8069` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:23:00.927892 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96315b58c9e4ed38c97c0d3b50e8069)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:23:00.928703 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e394f-a618-4f20-91f2-de5b468089b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:23:00.929311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e394f-a618-4f20-91f2-de5b468089b4 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:23:00.929589 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e394f-a618-4f20-91f2-de5b468089b4 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:23:00.929850 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:23:00.929967 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e394f-a618-4f20-91f2-de5b468089b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:23:00.931688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eb3e394f-a618-4f20-91f2-de5b468089b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:23:00.944181 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e394f-a618-4f20-91f2-de5b468089b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:23:00.945593 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e394f-a618-4f20-91f2-de5b468089b4 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:23:00.958047 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1444/2893] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:23:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:23:00.982718 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96315b58c9e4ed38c97c0d3b50e8069'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:23:00.984224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96315b58c9e4ed38c97c0d3b50e8069'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:23:00.984224 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96315b58c9e4ed38c97c0d3b50e8069'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:23:00.993905 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96315b58c9e4ed38c97c0d3b50e8069'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:23:01.006314 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96315b58c9e4ed38c97c0d3b50e8069'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:23:01.018713 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1450/2894] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:23:00 2022] DELETE /identity/v3/users/d96315b58c9e4ed38c97c0d3b50e8069 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:23:01.195794 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e394f-a618-4f20-91f2-de5b468089b4 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:23:01.207411 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e394f-a618-4f20-91f2-de5b468089b4 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:23:01.272612 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:23:01.273246 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:23:01.273610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:23:01.273826 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] PATH_INFO: `/v3/projects/c290ef5d11bd457faab2536a334b9bb7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:23:01.275036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c290ef5d11bd457faab2536a334b9bb7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:23:01.277081 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:23:01.316909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:23:01.327610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c290ef5d11bd457faab2536a334b9bb7', 'request_id': 'req-fa8bcc5a-ebcc-4c92-bc47-411055b17765'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:23:01.333660 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c290ef5d11bd457faab2536a334b9bb7', 'request_id': 'req-fa8bcc5a-ebcc-4c92-bc47-411055b17765'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:23:01.339373 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c290ef5d11bd457faab2536a334b9bb7', 'request_id': 'req-fa8bcc5a-ebcc-4c92-bc47-411055b17765'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:23:01.351523 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] The token cache is being invalidate because project c290ef5d11bd457faab2536a334b9bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:23:01.351815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:23:01.352437 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:23:01.353951 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1445/2895] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:23:01 2022] DELETE /identity/v3/projects/c290ef5d11bd457faab2536a334b9bb7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:23:11.015312 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:23:11.348470 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:23:17.759345 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:23:17.761069 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:23:17.818211 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97821f04-31b3-499c-8444-d2f7396ece65 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:23:17.865568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b68d7a3-0edd-4232-9dc5-3a2cdd70faa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:23:17.865926 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d7a3-0edd-4232-9dc5-3a2cdd70faa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:23:17.866234 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d7a3-0edd-4232-9dc5-3a2cdd70faa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:23:17.866495 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b68d7a3-0edd-4232-9dc5-3a2cdd70faa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:23:17.868509 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5b68d7a3-0edd-4232-9dc5-3a2cdd70faa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:23:17.941842 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68d7a3-0edd-4232-9dc5-3a2cdd70faa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:23:17.943319 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b68d7a3-0edd-4232-9dc5-3a2cdd70faa2 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:23:17.955835 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1451/2896] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:23:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:23:19.140812 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:23:19.142282 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:23:19.155974 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bcc5a-ebcc-4c92-bc47-411055b17765 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:23:19.159671 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dcce5-a240-4eae-99a1-decae3976c03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:23:19.160276 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dcce5-a240-4eae-99a1-decae3976c03 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:23:19.160553 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dcce5-a240-4eae-99a1-decae3976c03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:23:19.160825 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840dcce5-a240-4eae-99a1-decae3976c03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:23:19.162532 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-840dcce5-a240-4eae-99a1-decae3976c03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:23:19.176529 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dcce5-a240-4eae-99a1-decae3976c03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:23:19.177983 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840dcce5-a240-4eae-99a1-decae3976c03 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:23:19.189111 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1446/2897] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:23:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:23:21.492695 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a055a-2b65-4e0b-bba0-1a924cc28a81 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:23:21.493161 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a055a-2b65-4e0b-bba0-1a924cc28a81 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:23:21.493275 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3a055a-2b65-4e0b-bba0-1a924cc28a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:23:21.513376 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-da3a055a-2b65-4e0b-bba0-1a924cc28a81 None None] MFA Rules not processed for user `d2749b3c8da94ce1b4af6c708a2f9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:23:21.514626 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-da3a055a-2b65-4e0b-bba0-1a924cc28a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:23:21.571180 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1452/2898] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:23:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:23:29.184200 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:23:31.563866 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:24:26.947059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dcce5-a240-4eae-99a1-decae3976c03 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:24:26.956167 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840dcce5-a240-4eae-99a1-decae3976c03 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:24:26.959797 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f0514e-d44d-444d-b536-d8c8e0d368d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:24:26.960388 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0514e-d44d-444d-b536-d8c8e0d368d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:24:26.960663 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0514e-d44d-444d-b536-d8c8e0d368d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:24:26.960933 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f0514e-d44d-444d-b536-d8c8e0d368d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:24:26.962723 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-49f0514e-d44d-444d-b536-d8c8e0d368d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:24:26.973576 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0514e-d44d-444d-b536-d8c8e0d368d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:24:26.975021 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f0514e-d44d-444d-b536-d8c8e0d368d5 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:24:26.983380 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1447/2899] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:24:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 19 21:24:36.979456 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:25:34.494076 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a055a-2b65-4e0b-bba0-1a924cc28a81 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:25:34.495526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-da3a055a-2b65-4e0b-bba0-1a924cc28a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:25:34.552078 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3a055a-2b65-4e0b-bba0-1a924cc28a81 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:25:34.555930 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c5753-f6aa-4f98-bc6a-9d37b8011bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54d1ae400b5c4ae0955047b9e7750103', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:25:34.556534 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c5753-f6aa-4f98-bc6a-9d37b8011bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:25:34.556812 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c5753-f6aa-4f98-bc6a-9d37b8011bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:25:34.557084 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c5753-f6aa-4f98-bc6a-9d37b8011bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:25:34.558833 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a25c5753-f6aa-4f98-bc6a-9d37b8011bc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:25:34.570114 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c5753-f6aa-4f98-bc6a-9d37b8011bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:25:34.571531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c5753-f6aa-4f98-bc6a-9d37b8011bc8 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:25:34.581035 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1453/2900] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:25:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:25:44.576289 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=26 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:25:44.649251 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539933-d7e0-49a1-bc19-9e3d9b5d1dfd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:25:44.649859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539933-d7e0-49a1-bc19-9e3d9b5d1dfd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:25:44.649859 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c539933-d7e0-49a1-bc19-9e3d9b5d1dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:25:44.669288 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3c539933-d7e0-49a1-bc19-9e3d9b5d1dfd None None] MFA Rules not processed for user `d2749b3c8da94ce1b4af6c708a2f9db6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 19 21:25:44.672305 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c539933-d7e0-49a1-bc19-9e3d9b5d1dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:25:44.679815 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1448/2901] 10.222.0.16 () {58 vars in 958 bytes} [Fri Aug 19 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 19 21:25:45.004827 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c5753-f6aa-4f98-bc6a-9d37b8011bc8 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:25:45.014653 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c5753-f6aa-4f98-bc6a-9d37b8011bc8 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:25:45.019251 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd41b08-af15-4c2b-9118-2d2eb92788e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:25:45.020230 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd41b08-af15-4c2b-9118-2d2eb92788e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:25:45.020731 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd41b08-af15-4c2b-9118-2d2eb92788e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:25:45.021271 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd41b08-af15-4c2b-9118-2d2eb92788e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:25:45.024396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0dd41b08-af15-4c2b-9118-2d2eb92788e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:25:45.041290 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd41b08-af15-4c2b-9118-2d2eb92788e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:25:45.043851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd41b08-af15-4c2b-9118-2d2eb92788e6 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:25:45.052949 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1454/2902] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:25:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:25:45.502789 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c539933-d7e0-49a1-bc19-9e3d9b5d1dfd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:25:45.504307 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c539933-d7e0-49a1-bc19-9e3d9b5d1dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:25:45.583740 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c539933-d7e0-49a1-bc19-9e3d9b5d1dfd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:25:45.588803 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3cb5-4a23-4d3c-b8e9-c999c00459a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:25:45.589409 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3cb5-4a23-4d3c-b8e9-c999c00459a9 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:25:45.589688 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3cb5-4a23-4d3c-b8e9-c999c00459a9 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:25:45.589995 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d3cb5-4a23-4d3c-b8e9-c999c00459a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:25:45.591747 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e24d3cb5-4a23-4d3c-b8e9-c999c00459a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:25:45.603491 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d3cb5-4a23-4d3c-b8e9-c999c00459a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:25:45.605079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d3cb5-4a23-4d3c-b8e9-c999c00459a9 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:25:45.616069 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1449/2903] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:25:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:25:47.964758 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd41b08-af15-4c2b-9118-2d2eb92788e6 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:25:47.966886 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0dd41b08-af15-4c2b-9118-2d2eb92788e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:25:47.983754 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd41b08-af15-4c2b-9118-2d2eb92788e6 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:25:47.989783 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab070ad-90a1-4bb6-9253-b21ce1efb956 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:25:47.990396 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab070ad-90a1-4bb6-9253-b21ce1efb956 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:25:47.990670 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab070ad-90a1-4bb6-9253-b21ce1efb956 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:25:47.990939 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab070ad-90a1-4bb6-9253-b21ce1efb956 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:25:47.992660 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3ab070ad-90a1-4bb6-9253-b21ce1efb956 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:25:48.006237 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab070ad-90a1-4bb6-9253-b21ce1efb956 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:25:48.007825 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab070ad-90a1-4bb6-9253-b21ce1efb956 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:25:48.019338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1455/2904] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:25:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:25:55.611795 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=35 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:25:58.014115 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=12 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:25:59.783614 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3cb5-4a23-4d3c-b8e9-c999c00459a9 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:25:59.785016 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e24d3cb5-4a23-4d3c-b8e9-c999c00459a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:25:59.800926 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d3cb5-4a23-4d3c-b8e9-c999c00459a9 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:25:59.806452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac9fbd-e4ed-414a-8d00-0f6e94b5f343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:25:59.807059 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac9fbd-e4ed-414a-8d00-0f6e94b5f343 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:25:59.807339 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac9fbd-e4ed-414a-8d00-0f6e94b5f343 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:25:59.807639 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac9fbd-e4ed-414a-8d00-0f6e94b5f343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:25:59.809376 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7dac9fbd-e4ed-414a-8d00-0f6e94b5f343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:25:59.822036 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac9fbd-e4ed-414a-8d00-0f6e94b5f343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:25:59.823623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac9fbd-e4ed-414a-8d00-0f6e94b5f343 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:25:59.835468 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1450/2905] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:25:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:26:02.240466 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab070ad-90a1-4bb6-9253-b21ce1efb956 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:26:02.255486 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab070ad-90a1-4bb6-9253-b21ce1efb956 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:26:02.259346 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739221b-64ba-4f55-bd0a-221594dc8c13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:26:02.259980 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739221b-64ba-4f55-bd0a-221594dc8c13 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:26:02.260258 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739221b-64ba-4f55-bd0a-221594dc8c13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:26:02.260531 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739221b-64ba-4f55-bd0a-221594dc8c13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:26:02.262299 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3739221b-64ba-4f55-bd0a-221594dc8c13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:26:02.326443 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739221b-64ba-4f55-bd0a-221594dc8c13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:26:02.327931 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739221b-64ba-4f55-bd0a-221594dc8c13 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:26:02.338606 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1456/2906] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:26:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:26:09.830765 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:26:12.334124 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=30 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:26:27.732062 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac9fbd-e4ed-414a-8d00-0f6e94b5f343 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:26:27.793179 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac9fbd-e4ed-414a-8d00-0f6e94b5f343 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:26:27.798748 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e263-3ef6-4839-be0f-afabd9a1b09d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:26:27.799352 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e263-3ef6-4839-be0f-afabd9a1b09d service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:26:27.799630 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e263-3ef6-4839-be0f-afabd9a1b09d service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:26:27.799907 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a661e263-3ef6-4839-be0f-afabd9a1b09d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:26:27.801651 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a661e263-3ef6-4839-be0f-afabd9a1b09d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:26:27.835694 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661e263-3ef6-4839-be0f-afabd9a1b09d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:26:27.837272 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a661e263-3ef6-4839-be0f-afabd9a1b09d service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:26:27.847226 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1451/2907] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:26:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:26:37.842853 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=36 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:27:54.076908 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739221b-64ba-4f55-bd0a-221594dc8c13 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:27:54.135088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739221b-64ba-4f55-bd0a-221594dc8c13 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:27:54.141058 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3004ff7-3e4b-4667-b6dc-ee430afc8c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43a9ad5341b54c40a5427045660363c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:27:54.141688 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3004ff7-3e4b-4667-b6dc-ee430afc8c87 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:27:54.141995 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3004ff7-3e4b-4667-b6dc-ee430afc8c87 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:27:54.142266 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3004ff7-3e4b-4667-b6dc-ee430afc8c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:27:54.144728 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f3004ff7-3e4b-4667-b6dc-ee430afc8c87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:27:54.158265 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3004ff7-3e4b-4667-b6dc-ee430afc8c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:27:54.160174 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3004ff7-3e4b-4667-b6dc-ee430afc8c87 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:27:54.171696 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1457/2908] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:27:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:28:04.166998 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=23 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:28:07.345624 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e263-3ef6-4839-be0f-afabd9a1b09d service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:28:07.366549 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a661e263-3ef6-4839-be0f-afabd9a1b09d service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:28:07.370122 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb3b18-d021-4c61-82d3-dedbd1e7bba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b37a4275e7c6441aa9e8bb0298f56589', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:28:07.370715 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb3b18-d021-4c61-82d3-dedbd1e7bba4 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:28:07.370987 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb3b18-d021-4c61-82d3-dedbd1e7bba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:28:07.371255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb3b18-d021-4c61-82d3-dedbd1e7bba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:28:07.372980 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3edb3b18-d021-4c61-82d3-dedbd1e7bba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:28:07.459408 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb3b18-d021-4c61-82d3-dedbd1e7bba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:28:07.460841 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb3b18-d021-4c61-82d3-dedbd1e7bba4 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:28:07.480395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1452/2909] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:28:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:28:10.606351 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3004ff7-3e4b-4667-b6dc-ee430afc8c87 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:28:10.628798 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3004ff7-3e4b-4667-b6dc-ee430afc8c87 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:28:10.636608 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cb28c-5d97-48f9-b1c4-64f887034924 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:28:10.644111 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cb28c-5d97-48f9-b1c4-64f887034924 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:28:10.647047 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cb28c-5d97-48f9-b1c4-64f887034924 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:28:10.647602 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cb28c-5d97-48f9-b1c4-64f887034924 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:28:10.656749 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8e8cb28c-5d97-48f9-b1c4-64f887034924 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:28:10.697314 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cb28c-5d97-48f9-b1c4-64f887034924 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:28:10.699250 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cb28c-5d97-48f9-b1c4-64f887034924 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:28:10.711327 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1458/2910] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:28:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:28:13.780416 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb3b18-d021-4c61-82d3-dedbd1e7bba4 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:28:13.797346 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb3b18-d021-4c61-82d3-dedbd1e7bba4 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:28:13.803488 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef8467-c37f-403f-b121-bd02fc5a167f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:28:13.804160 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef8467-c37f-403f-b121-bd02fc5a167f service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:28:13.804452 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef8467-c37f-403f-b121-bd02fc5a167f service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:28:13.804717 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fef8467-c37f-403f-b121-bd02fc5a167f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:28:13.806610 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0fef8467-c37f-403f-b121-bd02fc5a167f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:28:13.822977 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef8467-c37f-403f-b121-bd02fc5a167f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:28:13.824382 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fef8467-c37f-403f-b121-bd02fc5a167f service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:28:13.836395 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1453/2911] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:28:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:28:13.851526 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cb28c-5d97-48f9-b1c4-64f887034924 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:28:13.852978 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8e8cb28c-5d97-48f9-b1c4-64f887034924 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:28:13.872757 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cb28c-5d97-48f9-b1c4-64f887034924 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:28:13.878632 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:28:13.880054 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:28:13.880454 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:28:13.881005 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] PATH_INFO: `/v3/users/d2749b3c8da94ce1b4af6c708a2f9db6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:28:13.882482 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2749b3c8da94ce1b4af6c708a2f9db6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:28:13.884872 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:28:13.940881 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2749b3c8da94ce1b4af6c708a2f9db6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:13.941561 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2749b3c8da94ce1b4af6c708a2f9db6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:13.942393 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2749b3c8da94ce1b4af6c708a2f9db6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:13.949851 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2749b3c8da94ce1b4af6c708a2f9db6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:13.963726 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2749b3c8da94ce1b4af6c708a2f9db6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:13.978068 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1459/2912] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:28:13 2022] DELETE /identity/v3/users/d2749b3c8da94ce1b4af6c708a2f9db6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:28:14.189530 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef8467-c37f-403f-b121-bd02fc5a167f service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:28:14.208335 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fef8467-c37f-403f-b121-bd02fc5a167f service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:28:14.272956 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:28:14.273599 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:28:14.273909 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:28:14.274192 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] PATH_INFO: `/v3/projects/b5c70daaa3b340a286f8f4ebe26b807d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:28:14.275356 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5c70daaa3b340a286f8f4ebe26b807d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:28:14.277536 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:28:14.322156 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:28:14.325260 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5c70daaa3b340a286f8f4ebe26b807d', 'request_id': 'req-b8f95a08-894e-4532-9165-26211ef445ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.333527 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5c70daaa3b340a286f8f4ebe26b807d', 'request_id': 'req-b8f95a08-894e-4532-9165-26211ef445ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.339195 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5c70daaa3b340a286f8f4ebe26b807d', 'request_id': 'req-b8f95a08-894e-4532-9165-26211ef445ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.352079 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] The token cache is being invalidate because project b5c70daaa3b340a286f8f4ebe26b807d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:28:14.352311 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.353002 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.354627 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1454/2913] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:28:14 2022] DELETE /identity/v3/projects/b5c70daaa3b340a286f8f4ebe26b807d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:28:14.358877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:28:14.360179 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:28:14.374338 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d7f31-1d9c-487f-bf7f-c7a72c4c40c7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:28:14.431877 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:28:14.432507 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:28:14.432784 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:28:14.433052 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] PATH_INFO: `/v3/users/4320be012bb045afb47948c6c7217a70` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:28:14.434635 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4320be012bb045afb47948c6c7217a70)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:28:14.436631 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:28:14.487499 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4320be012bb045afb47948c6c7217a70'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.488363 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4320be012bb045afb47948c6c7217a70'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.489558 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4320be012bb045afb47948c6c7217a70'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.497939 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4320be012bb045afb47948c6c7217a70'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.503426 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:28:14.512210 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4320be012bb045afb47948c6c7217a70'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.526806 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1460/2914] 10.222.0.16 () {58 vars in 1224 bytes} [Fri Aug 19 21:28:14 2022] DELETE /identity/v3/users/4320be012bb045afb47948c6c7217a70 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:28:14.699208 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:28:14.700656 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:28:14.715623 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f95a08-894e-4532-9165-26211ef445ef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:28:14.779665 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3476c91ad03941919217e0940471d201', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4f484db3eb0540218c8a18fc8601facf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:28:14.780291 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:28:14.780567 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:28:14.781147 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] PATH_INFO: `/v3/projects/1c51d69ad366402f977fea46158bcfa4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:28:14.782037 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c51d69ad366402f977fea46158bcfa4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:28:14.783952 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:28:14.835315 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c51d69ad366402f977fea46158bcfa4', 'request_id': 'req-aa423e81-604e-4a5f-8d77-d58dd544c1da'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.845625 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c51d69ad366402f977fea46158bcfa4', 'request_id': 'req-aa423e81-604e-4a5f-8d77-d58dd544c1da'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.854561 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c51d69ad366402f977fea46158bcfa4', 'request_id': 'req-aa423e81-604e-4a5f-8d77-d58dd544c1da'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.867968 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] The token cache is being invalidate because project 1c51d69ad366402f977fea46158bcfa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 19 21:28:14.868255 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.868901 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aa423e81-604e-4a5f-8d77-d58dd544c1da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 19 21:28:14.870425 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1455/2915] 10.222.0.16 () {58 vars in 1233 bytes} [Fri Aug 19 21:28:14 2022] DELETE /identity/v3/projects/1c51d69ad366402f977fea46158bcfa4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 19 21:28:24.522703 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:28:24.864732 dv-cinder-852209-6-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=11,keystone:SELECT=58,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 19 21:31:34.652755 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 19 21:31:34.654650 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:31:34.723982 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d0036-6bf8-43e4-a2f4-6f7a8905833b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 19 21:31:34.784088 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cd4b9-0e8a-407f-8a5d-b6584ccf27bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '824165e55d2f4742aa435f1edff9b403', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '052c34fcd80c4f0e934fb5cdc46cc60b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 19 21:31:34.784712 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cd4b9-0e8a-407f-8a5d-b6584ccf27bb service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 19 21:31:34.785057 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cd4b9-0e8a-407f-8a5d-b6584ccf27bb service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 19 21:31:34.785334 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cd4b9-0e8a-407f-8a5d-b6584ccf27bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 19 21:31:34.787100 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9b9cd4b9-0e8a-407f-8a5d-b6584ccf27bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 19 21:31:34.798958 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cd4b9-0e8a-407f-8a5d-b6584ccf27bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 19 21:31:34.800568 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cd4b9-0e8a-407f-8a5d-b6584ccf27bb service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 19 21:31:34.811188 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1461/2916] 10.222.0.16 () {60 vars in 1365 bytes} [Fri Aug 19 21:31:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 19 21:31:44.806165 dv-cinder-852209-6-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=58 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}